KlingTeam / 3DTrajMaster

3DTrajMaster-CogVideoX

Introduction

3DTrajMaster controls one or multiple entity motions in 3D space with entity-specific 3D trajectories for text-to-video (T2V) generation. It has the following features::

• 6 Domain of Freedom (DoF) : control 3D entity location and orientation.
• Diverse Entities : human, animal, robot, car, even abstract fire, breeze, etc.
• Diverse Background : city, forest, desert, gym, sunset beach, glacier, hall, night city, etc.
• Complex 3D trajectories : 3D occlusion, rotating in place, 180°/continuous 90° turnings, etc.
• Fine-grained Entity Prompt : change human hair, clothing, gender, figure size, accessory, etc.

Usage

This is the implementation based on CogVideoX-5B. Please refer to our github for details on usage.

Citation

If you find this project useful, please consider citing:

@inproceedings{fu20243dtrajmaster,
    author    = {Fu, Xiao and Liu, Xian and Wang, Xintao and Peng, Sida and Xia, Menghan and Shi, Xiaoyu and Yuan, Ziyang and Wan, Pengfei and Zhang, Di and Lin, Dahua},
    title     = {3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation},
    booktitle = {ICLR},
    year      = {2025}
}