episod / tt-animatediff-container-v5.1

tt-animatediff

AnimateDiff text-to-video on Tenstorrent Blackhole: a TTNN Stable Diffusion 1.4 UNet with cross-frame temporal attention, served over HTTP as 512x512 GIF loops. Offers style LoRAs, LCM-LoRA fast sampling, prompt travel, latent chaining, per-step previews and an optional MotionAdapter.

Runs on p150 or p150 — see the serve profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

Direct use: Short looping text-to-video clips, style experiments with SD 1.5 LoRAs, and fast previews with LCM.

Out-of-scope use: Image-to-video, resolutions other than 512x512, multi-chip or multi-user throughput, and SDXL checkpoints or LoRAs.

Quickstart

uv tool install tenstorrent   # once — the Tenstorrent CLI, `tt`
tt model pull episod/tt-animatediff-container-v5.1
tt serve episod/tt-animatediff-container-v5.1

tt model pull (or tt-model pull --with-weights ) downloads the Docker image and the CompVis/stable-diffusion-v1-4 weights (into your HF cache; they are not in the image). tt serve (or tt-model serve ) starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete .

Without tt-cli — tt-model alone does the whole job:

tt-model pull  episod/tt-animatediff-container-v5.1 --with-weights
tt-model serve episod/tt-animatediff-container-v5.1

Serve profiles

One image serves every profile below; pick one with --profile .

Using it

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes above for the payload it expects.

POST /v1/videos/generations returns a base64 GIF. Fields beyond prompt , num_frames , num_inference_steps , guidance_scale and seed : lora (a name from GET /v1/models ), prompt_schedule , sampler ( pndm or euler ), chain_save / chain_from / chain_alpha , motion_adapter (+ _alpha , _skip ), preview / preview_every (read progress from GET /v1/videos/preview ) and temporal_alpha . The full profile (default) offers the pixel , pixel-fast and lcm LoRA variants and the motion adapter; --profile plain offers neither. A variant that includes the LCM-LoRA defaults to 4 steps and guidance 1.5 when the request leaves them out.

Expected performance

One Blackhole chip of a P300c pair, 8 frames at 512x512, warm, median of 3: 15.3 s at 25 steps, 12.8 s at 20, 8.7 s at 12. A style LoRA costs +0.5 s. An LCM variant runs 4 steps in 4.1 s (3.7x faster than 25 steps); style + LCM takes 4.3 s. Prompt schedule +0.2 s, chaining +0.1 s, per-step preview +4.9 s at 20 steps. The first request after start compiles kernels and took 119.6 s. Measured through this container image with scripts/benchmark_modes.py (docs/LORA.md, "Every served mode, timed").

Limitations

One chip, 512x512 only. LoRAs must be single kohya-format SD 1.5 UNet files; SDXL, LoHa, LoKr and DoRA files are refused at startup. LCM clips are less frame-coherent than 25-step clips. The MotionAdapter takes about 24 s for 2 frames at 6 steps with nothing skipped (alpha 0 is just as slow) and 3.4 s with up1 and up2 skipped. The first request to each LoRA variant is about 1.4 s slower than the following ones. No CPU, simulator or Lightning-UNet modes here.

Risks and safety considerations

The full profile downloads third-party LoRA files and the motion adapter from the Hugging Face Hub at startup, and a file that fails to load stops the server before it reports ready. Check each LoRA's own license before use. Output is AI-generated video and can reproduce biases of the SD 1.4 training data.

Licensing

The weights are CompVis/stable-diffusion-v1-4 under the CreativeML OpenRAIL-M license, fetched at start and not bundled. The TTNN port and serving code in code/ are Apache-2.0. LoRA files carry their own licenses.

Related packages

Source and docs at https://github.com/tenstorrent/tt-animatediff . The diffusers custom pipeline and v6 thin bundle are episod/tt-animatediff .

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/episod/tt-animatediff-container-v5.1/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected] .

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: