lerobot / pi0_base

π₀ (Pi0)

π₀ is a Vision-Language-Action model for general robot control , from Physical Intelligence. The LeRobot implementation is adapted from their open source OpenPI repository.

Model Overview

π₀ represents a breakthrough in robotics as the first general-purpose robot foundation model developed by Physical Intelligence . Unlike traditional robots that are narrow specialists programmed for repetitive motions, π₀ is designed to be a generalist policy that can understand visual inputs, interpret natural language instructions, and control a variety of different robots across diverse tasks.

The Vision for Physical Intelligence

As described by Physical Intelligence, while AI has achieved remarkable success in digital domains, from chess-playing to drug discovery, human intelligence still dramatically outpaces AI in the physical world. To paraphrase Moravec's paradox, winning a game of chess represents an "easy" problem for AI, but folding a shirt or cleaning up a table requires solving some of the most difficult engineering problems ever conceived. π₀ represents a first step toward developing artificial physical intelligence that enables users to simply ask robots to perform any task they want, just like they can with large language models.

Architecture and Approach

π₀ combines several key innovations:

• Flow Matching : Uses a novel method to augment pre-trained VLMs with continuous action outputs via flow matching (a variant of diffusion models)
• Cross-Embodiment Training : Trained on data from 8 distinct robot platforms including UR5e, Bimanual UR5e, Franka, Bimanual Trossen, Bimanual ARX, Mobile Trossen, and Mobile Fibocom
• Internet-Scale Pre-training : Inherits semantic knowledge from a pre-trained 3B parameter Vision-Language Model
• High-Frequency Control : Outputs motor commands at up to 50 Hz for real-time dexterous manipulation

Training

For training π₀, you can use the standard LeRobot training script with the appropriate configuration:

python src/lerobot/scripts/train.py \
    --dataset.repo_id=your_dataset \
    --policy.type=pi0 \
    --output_dir=./outputs/pi0_training \
    --job_name=pi0_training \
    --policy.pretrained_path=pepijn223/pi0_base \
    --policy.repo_id=your_repo_id \
    --policy.compile_model=true \
    --policy.gradient_checkpointing=true \
    --policy.dtype=bfloat16 \
    --steps=3000 \
    --policy.scheduler_decay_steps=3000 \
    --policy.device=cuda \
    --batch_size=32

Conversion Details

This model was converted from JAX to PyTorch using the OpenPI conversion script:

python examples/convert_jax_model_to_pytorch.py \
    --checkpoint_dir /pi0_base \
    --config_name pi0_base \
    --output_path /pi0_base/pytorch/fp32/ \
    --precision float32

Citation

If you use this model, please cite the original OpenPI work:

@article{openpi2024,
    title={Open-World Robotic Manipulation with Vision-Language-Action Models},
    author={Physical Intelligence},
    year={2024},
    url={https://github.com/Physical-Intelligence/openpi}
}

Original Repository

OpenPI GitHub Repository

License

This model follows the same license as the original OpenPI repository.