nvidia / GEN3C-Cosmos-7B

GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control

CVPR 2025 (Highlight)

Xuanchi Ren* , Tianchang Shen* Jiahui Huang , Huan Ling , Yifan Lu , Merlin Nimier-David , Thomas Müller , Alexander Keller , Sanja Fidler , Jun Gao
* indicates equal contribution

Description:

GEN3C is a generative video model with precise camera control and temporal three-dimensional (3D) Consistency. We achieve this with a 3D cache: point clouds obtained by predicting the pixel-wise depth of seed images or previously generated frames. When generating the next frames, GEN3C is conditioned on the two-dimensional (2D) renderings of the 3D cache with the new camera trajectory provided by the user. Our results demonstrate more precise camera control than prior work, as well as state-of-the-art results in sparse-view novel view synthesis, even in challenging settings such as driving scenes and monocular dynamic video.

This model is ready for commercial/non-commercial use

License/Terms of Use:

This model is released under the NVIDIA Open Model License . For a custom license, please contact [email protected] .

Important Note: If you bypass, disable, reduce the efficacy of, or circumvent any technical limitation, safety guardrail or associated safety guardrail hyperparameter, encryption, security, digital rights management, or authentication mechanism contained in the Model, your rights under NVIDIA Open Model License Agreement will automatically terminate.

Deployment Geography:

Global

Use Case:

This model is intended for researchers interested in developing consistent video generation and allows users to use cameras to control the final generation. For AV applications, we can enable users to generate driving videos and specify the camera trajectories in this video, such as switching from the viewpoint of a sedan car to a truck, or looking at a different lane.

Release Date:

Github 06/10/2025 via https://github.com/nv-tlabs/Gen3C

Reference:

GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control Paper , Project Page

Model Architecture:

Architecture Type: Convolutional Neural Network (CNN), Transformer
Network Architecture: Transformer

**This model was developed based on Cosmos Predict 1
** This model has 7B of model parameters.

Input:

Input Type(s): Camera Parameters, Image
Input Format(s): 1D Array of Camera Poses, 2D Array of Images.
Input Parameters: Camera Poses (1D), Images (2D)
Other Properties Related to Input: The input image should be 720 * 1080 resolution, and we recommend using 121 frames for the camera parameters.

Output:

Output Type(s): Videos
Output Format: MP4 video
Output Parameters: 3D (N x H x W), with 3 channels (Red, Green, Blue ((RGB))
Other Properties Related to Output: A sequence of images (N x H x W x 3), N is the number of frames, H is the height and W is the width. Three (3) refers to the number of RGB channels.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems A100 and H100. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):
* Cosmos-Predict1

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
  
• NVIDIA Blackwell
  
• NVIDIA Hopper
  

[Preferred/Supported] Operating System(s):

• Linux

Model Version(s):

-V1.0

Inference:

Engine: Cosmos-Predict1 Test Hardware:

• NVIDIA Ampere
  
• NVIDIA Blackwell
  
• NVIDIA Hopper
  

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Users are responsible for model inputs and outputs. Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms, prior to deployment.

For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards link to subcard .

Please report security vulnerabilities or NVIDIA AI Concerns here .

Plus Plus (++) Promise

We value you, the datasets, the diversity they represent, and what we have been entrusted with. This model and its associated data have been:

• Verified to comply with current applicable disclosure laws, regulations, and industry standards.
• Verified to comply with applicable privacy labeling requirements.
• Annotated to describe the collector/source (NVIDIA or a third-party).
• Characterized for technical limitations.
• Reviewed to ensure proper disclosure is accessible to, maintained for, and in compliance with NVIDIA data subjects and their requests.
• Reviewed before release.
• Tagged for known restrictions and potential safety implications.

Bias

Explainability

Privacy

Safety

Citation

 @inproceedings{ren2025gen3c,
    title={GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control},
    author={Ren, Xuanchi and Shen, Tianchang and Huang, Jiahui and Ling, Huan and 
        Lu, Yifan and Nimier-David, Merlin and Müller, Thomas and Keller, Alexander and 
        Fidler, Sanja and Gao, Jun},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2025}
}