episod / tt-openvla

tt-openvla

A TT-Metal / TTNN bring-up of OpenVLA -7B -- a vision-language-action model for robot manipulation, combining a fused DINOv2 + SigLIP vision backbone with a LLaMA-2-7B language model -- for Tenstorrent Blackhole hardware. Code: tsingletaryTT/tt-openvla .

No separate checkpoint is hosted here. The port uses the real, original openvla/openvla-7b weights directly -- no stripping, conversion, or fine-tuning was done, so there's no new derived artifact that isn't already correctly hosted and licensed at the source. This repo is a model card pointing at the implementation and the real checkpoint it runs, not a weights mirror.

What's validated

Every stage checked against a real reference implementation, each at PCC >= 0.995 except where noted:

• DINOv2 ViT-L/14, register-token variant (the exact tower OpenVLA actually uses, not the plain checkpoint) -- PCC 0.9992
• SigLIP ViT-So400M/14 -- PCC 0.9967
• Fused DINOv2+SigLIP vision backbone (matching OpenVLA's real fusion mechanism) -- PCC 0.9986
• Projector MLP, validated against the real fine-tuned weights -- PCC 0.99999
• Full 32-layer LLaMA-2-7B backbone, real fine-tuned weights, reusing tt-metal's own tt_transformers attention/RoPE/KV-cache kernels -- PCC 0.9966
• End-to-end "Grounded Check" : a real image + OpenVLA's own documented prompt format, through the whole pipeline -- vision backbone, fused embeddings, one PREFILL pass plus 6 real autoregressive DECODE steps across all 32 real LLaMA layers, on a real 2-device Blackhole mesh -- producing an actual, deterministic decoded 7-DoF action from real weights.

See the GitHub repo 's tt/ directory for the individual correctness tests and tt/demo_grounded_check.py for the end-to-end demo, including notes on real bugs found and fixed along the way (a tile-alignment issue in SigLIP's attention on this hardware, and a non-determinism bug in the decode loop's output-tile handling).

Try it

gradio_app/app.py is an interactive demo -- upload an image, write an instruction, get back a real decoded 7-DoF action from an actual forward pass (not a scripted response). Runs against real Blackhole hardware by default, or --backend reference for CPU-only. Also registered with tt-discolike ( .disco/app.yaml ) for one-click start/stop, and has a tt-model.yaml container manifest for tt-model-manager (validates cleanly; the actual container build is currently blocked on an environment issue -- see the GitHub repo's README for the exact, resumable state).

Benchmarks

Real end-to-end latency for the full pipeline, measured on this port's own hardware ( tt/benchmark.py / tt/cpu_benchmark.py ):

License

This port's own code is MIT, matching openvla/openvla 's license. The openvla/openvla-7b checkpoint itself is a fine-tune of Meta's Llama-2-7B, so using those weights is subject to the Llama Community License separately from this repo's own MIT terms.