facebook / sam-3d-body-dinov3

SAM 3D Body: Robust Full-Body Human Mesh Recovery

Meta Superintelligence Labs

SAM 3D Body (3DB) is a promptable model for single-image full-body 3D human mesh recovery (HMR). Our method demonstrates state-of-the-art performance, with strong generalization and consistent accuracy in diverse in-the-wild conditions. 3DB estimates the human pose of the body, feet, and hands based on the Momentum Human Rig (MHR), a new parametric mesh representation that decouples skeletal structure and surface shape for improved accuracy and interpretability.

3DB employs an encoder-decoder architecture and supports auxiliary prompts, including 2D keypoints and masks, enabling user-guided inference similar to the SAM family of models. Our model is trained on high-quality annotations from a multi-stage annotation pipeline using differentiable optimization, multi-view geometry, dense keypoint detection, and a data engine to collect and annotated data covering both common and rare poses across a wide range of viewpoints.

Key Features

• Robust Full-Body Performance : Superior handling of occlusions, hard poses, and challenging viewpoints
• Promptable Model : Supports auxiliary prompts including 2D keypoints and masks for user-guided inference
• Momentum Human Rig (MHR) : New parametric mesh representation that decouples skeletal structure and surface shape
• Large-Scale and High-Quality Data : Multi-stage annotation pipeline for large-scale, diverse, and high-quality data

Quick Start

Installation

Please refer to https://github.com/facebookresearch/sam-3d-body/blob/main/INSTALL.md for detailed installation guidelines.

Inference

from sam_3d_body import load_sam_3d_body_hf, SAM3DBodyEstimator

# Load model from HuggingFace
model, model_cfg = load_sam_3d_body_hf("facebook/sam-3d-body-dinov3")

# Create estimator
estimator = SAM3DBodyEstimator(
    sam_3d_body_model=model,
    model_cfg=model_cfg,
)

# 3D human mesh recovery
outputs = estimator.process_one_image("path/to/image.jpg")

Output Format

Each detected person returns a dictionary containing:

• pred_vertices : 3D mesh vertices in camera coordinates
• pred_keypoints_3d : 3D pose keypoints
• pred_keypoints_2d : 2D pose keypoints projected to image
• pred_cam_t : Camera translation parameters
• focal_length : Estimated focal length
• body_pose_params : Body pose parameters
• hand_pose_params : Hand pose parameters
• shape_params : Body shape parameters

Citation

If you use SAM 3D Body or the SAM 3D Body dataset in your research, please use the following BibTeX entry.

@article{yang2025sam3dbody,
  title={SAM 3D Body: Robust Full-Body Human Mesh Recovery},
  author={Yang, Xitong and Kukreja, Devansh and Pinkus, Don and Sagar, Anushka and Fan, Taosha and Park, Jinhyung and Shin, Soyong and Cao, Jinkun and Liu, Jiawei and Ugrinovic, Nicolas and Feiszli, Matt and Malik, Jitendra and Dollar, Piotr and Kitani, Kris},
  journal={arXiv preprint; identifier to be added},
  year={2025}
}

License

The SAM 3D Body model is licensed under SAM License .

Links

• Paper : https://ai.meta.com/research/publications/sam-3d-body-robust-full-body-human-mesh-recovery/
• Code : https://github.com/facebookresearch/sam-3d-body
• Demo : https://www.aidemos.meta.com/segment-anything/editor/convert-body-to-3d
• Website : https://ai.meta.com/sam3d/
• Dataset : https://huggingface.co/datasets/facebook/sam-3d-body-dataset