NVIDIA unveiled the new NVIDIA Cosmos Predict-2 foundation model, intended to accelerate the development of the next generation of autonomous vehicle (AV) architecture. This model, part of the NVIDIA Cosmos platform, features advanced capabilities for predicting the future state of the world and generating physics-based synthetic data. Leading companies like Uber, Plus, and Oxa are already leveraging this technology to rapidly scale their data production.
The Cosmos Predict-2 model boasts a deeper understanding of text and visual inputs compared to its predecessor, resulting in a significant reduction in visual hallucinations and increased detail in generated videos. By utilizing NVIDIA GB200 NVL72 systems and the DGX Cloud platform, this model has dramatically accelerated the production of synthetic data.
One of the key introduced capabilities is the ability to convert single-view videos (such as those from standard dashboard cameras) into multi-view videos. This feature enables developers to leverage vast existing data repositories to train driving models. Additionally, new tools like Cosmos Transfer NIM and NuRec Fixer have been introduced to address gaps in reconstructed data.
NVIDIA also announced that the open-source CARLA simulator will integrate these new tools into its latest version. This collaboration will allow over 150,000 developers worldwide to simulate driving scenarios with varied weather and lighting conditions using simple text prompts with high accuracy.