CompVis / ZipMo

ZipMo (Learning Long-term Motion Embeddings for Efficient Kinematics Generation)

ZipMo is a motion-space model for efficient long-horizon kinematics generation. It learns compact long-term motion embeddings from large-scale tracker-derived trajectories and generates plausible future motion directly in this learned motion space. The model supports spatial-poke conditioning for open-domain videos and task/text-embedding conditioning for LIBERO robotics evaluation.

Paper and Abstract

ZipMo was introduced in the CVPR 2026 paper Learning Long-term Motion Embeddings for Efficient Kinematics Generation .

Understanding and predicting motion is a fundamental component of visual intelligence. Although video models can synthesize scene dynamics, exploring many possible futures through full video generation is expensive. ZipMo instead operates directly on long-term motion embeddings learned from tracker trajectories, enabling efficient generation of long, realistic motions while preserving dense reconstruction at arbitrary spatial query points.

ZipMo generates long-horizon motion in a compact learned motion space, supporting spatial-poke conditioning for open-domain videos and task-conditioned action prediction on LIBERO.

Usage

For programmatic use, the simplest way to use ZipMo is via torch.hub :

import torch

repo = "CompVis/long-term-motion"

# Open-domain motion prediction
planner_sparse = torch.hub.load(repo, "zipmo_planner_sparse")
planner_dense = torch.hub.load(repo, "zipmo_planner_dense")

# Motion autoencoder
vae = torch.hub.load(repo, "zipmo_vae")

LIBERO planning and policy components can be loaded in the same way:

import torch

repo = "CompVis/long-term-motion"

# LIBERO planners
libero_atm_planner = torch.hub.load(repo, "zipmo_planner_libero", "atm")
libero_tramoe_planner = torch.hub.load(repo, "zipmo_planner_libero", "tramoe")

# LIBERO policy heads
policy_head_atm = torch.hub.load(repo, "zipmo_policy_head", "atm")
policy_head_tramoe_goal = torch.hub.load(repo, "zipmo_policy_head", "tramoe", "goal")

Available Torch Hub entries:

• zipmo_planner_sparse : sparse-poke planner for open-domain motion prediction.
• zipmo_planner_dense : dense-conditioning planner for open-domain motion prediction.
• zipmo_vae : long-term motion autoencoder.
• zipmo_planner_libero : LIBERO planner with mode atm or tramoe .
• zipmo_policy_head : LIBERO policy head with mode atm or tramoe . For tramoe , pass one of 10 , goal , object , or spatial .

For the interactive demo, standard track prediction evaluation, LIBERO rollout evaluation, and training instructions, see the GitHub repository .

Citation

If you find our model or code useful, please cite our paper:

@inproceedings{stracke2026motionembeddings,
  title     = {Learning Long-term Motion Embeddings for Efficient Kinematics Generation},
  author    = {Stracke, Nick and Bauer, Kolja and Baumann, Stefan Andreas and Bautista, Miguel Angel and Susskind, Josh and Ommer, Bj{\"o}rn},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year      = {2026}
}