Lightricks / LTX-2.5-Diffusers

LTX-2.5 — Video, Audio & World Simulation

Full control and customization — self-host on your infrastructure.

Homepage Docs GitHub Research API Playground Discord

LTX License

LTX-2.5 — Video, Audio & World Simulation

Full control and customization — self-host on your infrastructure.

Homepage Docs GitHub Research API Playground Discord

LTX License

Diffusers weights for LTX-2.5 .

LTX-2.5 is an open world model with open weights, built for local execution and fine-tuning. Its established use is generating synchronized, high-fidelity video and audio from text, image, and video inputs; applicability to emerging domains such as robotics and physical AI is developing.

Full control and customization — self-host on your own infrastructure. No per-generation billing, no per-seat lock-in, no forced API dependency. Revenue is measured across the whole entity, including subsidiaries and affiliates under common control. The full, binding terms live in LICENSE .

Layout

Encoding always uses vae/ , and LTX2Pipeline decodes with vae/ too. The diffusion decoder is a diffusion model in its own right rather than a pipeline component, so it is driven by LTX2VideoDiffusionDecodePipeline : run the pipeline with output_type="latent" , then decode. Both decoders consume the same latents.

scheduler/ is configured for the distilled transformer ( use_dynamic_shifting: false , shift_terminal: null ) so that distilled sigma schedules are used exactly as given. See Full / SFT transformer for the override the full DiT needs.

Install

LTX-2.5 support is not in a diffusers release yet, so install from main:

pip install git+https://github.com/huggingface/diffusers

Downloading only what you need

from_pretrained fetches just the components listed in model_index.json , so the distilled path never pulls transformer_full/ . If you snapshot the repo instead, exclude it explicitly — 72 GB rather than 110 GB:

hf download Lightricks/LTX-2.5-Diffusers --exclude "transformer_full/*"

Quick start — distilled, convolutional decode

Distilled inference is driven by an explicit sigma schedule, not a step count, and runs unguided — guidance_scale=1.0 plus STG and modality guidance zeroed, since the pipeline defaults are the SFT values. Passing num_inference_steps instead would hand the model a generic linear schedule and quietly cost quality.

import torch
from diffusers import LTX2Pipeline
from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES
from diffusers.utils import encode_video

MODEL_ID = "Lightricks/LTX-2.5-Diffusers"

pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

video, audio = pipe(
    prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, "
           "the camera tracking alongside, snow crunching underfoot.",
    negative_prompt=DEFAULT_NEGATIVE_PROMPT,
    width=960,
    height=544,
    num_frames=121,
    frame_rate=24.0,
    sigmas=DISTILLED_SIGMA_VALUES,
    guidance_scale=1.0,
    audio_guidance_scale=1.0,
    stg_scale=0.0,
    audio_stg_scale=0.0,
    modality_scale=1.0,
    audio_modality_scale=1.0,
    generator=torch.Generator("cuda").manual_seed(42),
    output_type="np",
    return_dict=False,
)

encode_video(
    video[0],
    fps=24,
    output_path="ltx25.mp4",
    audio=audio[0].float().cpu(),
    audio_sample_rate=pipe.vocoder.config.output_sampling_rate,
)

Video and audio are generated jointly. Read the audio rate off the vocoder rather than hardcoding it — this checkpoint ships LTX2VocoderWithBWE , whose output_sampling_rate is 48000.

Two-stage distilled generation

Better quality than the single-stage quick start: half resolution, x2 latent upsample, then a 3-sigma tail at full resolution. One generator across both calls, so stage 2 continues the noise stream.

import torch
from diffusers import LTX2LatentUpsamplePipeline, LTX2Pipeline
from diffusers.pipelines.ltx2.latent_upsampler import LTX2LatentUpsamplerModel
from diffusers.pipelines.ltx2.utils import (
    DEFAULT_NEGATIVE_PROMPT,
    DISTILLED_SIGMA_VALUES,
    STAGE_2_DISTILLED_SIGMA_VALUES,
)
from diffusers.utils import encode_video

MODEL_ID = "Lightricks/LTX-2.5-Diffusers"
# Stage 1 resolution; stage 2 runs at 2x this.
HEIGHT, WIDTH, NUM_FRAMES, FRAME_RATE = 544, 960, 121, 24.0

pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()
pipe.vae.enable_tiling()  # stage 2 decodes at 2x

latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained(
    MODEL_ID, subfolder="latent_upsampler", dtype=torch.bfloat16
).to("cuda")
upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler)

generator = torch.Generator("cuda").manual_seed(42)
shared = dict(
    prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, "
           "the camera tracking alongside, snow crunching underfoot.",
    negative_prompt=DEFAULT_NEGATIVE_PROMPT,
    frame_rate=FRAME_RATE,
    guidance_scale=1.0,
    audio_guidance_scale=1.0,
    stg_scale=0.0,
    audio_stg_scale=0.0,
    modality_scale=1.0,
    audio_modality_scale=1.0,
    generator=generator,
    return_dict=False,
)

stage_1_latents, audio_latents = pipe(
    height=HEIGHT, width=WIDTH, num_frames=NUM_FRAMES,
    sigmas=DISTILLED_SIGMA_VALUES, output_type="latent", **shared,
)

upsampled_latents = upsample_pipe(
    latents=stage_1_latents, output_type="latent", return_dict=False
)[0]

# Stage 2 takes its size from the upsampled latents, so pass no height/width.
video, audio = pipe(
    num_frames=NUM_FRAMES,
    sigmas=STAGE_2_DISTILLED_SIGMA_VALUES,
    latents=upsampled_latents,
    audio_latents=audio_latents,
    noise_scale=STAGE_2_DISTILLED_SIGMA_VALUES[0],
    output_type="np",
    **shared,
)

encode_video(
    video[0],
    fps=int(FRAME_RATE),
    output_path="ltx25_two_stage.mp4",
    audio=audio[0].float().cpu(),
    audio_sample_rate=pipe.vocoder.config.output_sampling_rate,
)

To finish this with the diffusion decoder instead, ask stage 2 for output_type="latent" and follow the next section — including its by-hand audio decode, since latents skip the vocoder.

Decode with the diffusion decoder

Ask the pipeline for latents, then decode them. Two things differ from the snippet above: output_type="latent" skips the vocoder, so the audio comes back as latents and has to be finished by hand; and the decoder needs pip install kernels , which fetches the NATTEN kernel from the Hub (a local natten install is not used).

import torch
from diffusers import LTX2Pipeline, LTX2VideoDiffusionDecoderModel
from diffusers.models.autoencoders.ltx2_diffusion_decoder import LTX2VideoVaeNeighborhoodNattenProcessor
from diffusers.pipelines.ltx2.pipeline_ltx2_diffusion_decode import LTX2VideoDiffusionDecodePipeline
from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES
from diffusers.utils import encode_video

MODEL_ID = "Lightricks/LTX-2.5-Diffusers"

pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()
generator = torch.Generator("cuda").manual_seed(42)

latents, audio_latents = pipe(
    prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, "
           "the camera tracking alongside, snow crunching underfoot.",
    negative_prompt=DEFAULT_NEGATIVE_PROMPT,
    width=960, height=544, num_frames=121, frame_rate=24.0,
    sigmas=DISTILLED_SIGMA_VALUES,
    guidance_scale=1.0,
    audio_guidance_scale=1.0,
    stg_scale=0.0,
    audio_stg_scale=0.0,
    modality_scale=1.0,
    audio_modality_scale=1.0,
    generator=generator,
    output_type="latent",
    return_dict=False,
)

# `output_type="latent"` skips the vocoder, so finish the audio here. These latents are already
# denormalized, which is what `audio_vae.decode` expects.
mel = pipe.audio_vae.decode(audio_latents.to(pipe.audio_vae.dtype), return_dict=False)[0]
audio = pipe.vocoder(mel)

decoder = LTX2VideoDiffusionDecoderModel.from_pretrained(
    MODEL_ID, subfolder="diffusion_decoder", dtype=torch.bfloat16
).to("cuda")

# The default FlexAttention processor is uncompiled, so it materialises the full score matrix and needs
# tens of GB at video resolutions. NATTEN's kernels are what the original implementation uses.
# `set_attention_backend` is not an alternative: no backend other than `flex` accepts the BlockMask.
# Every attention module in the decoder is the same neighborhood attention, so one call swaps them all.
decoder.set_attn_processor(LTX2VideoVaeNeighborhoodNattenProcessor())
# Decode in overlapping tiles so peak memory is bounded by the tile size, not the video size.
decoder.enable_tiling()

decode_pipe = LTX2VideoDiffusionDecodePipeline(diffusion_decoder=decoder, scheduler=pipe.scheduler)

# denormalize=False: `output_type="latent"` already applied the latent statistics, so applying them
# again would rescale every channel by its std a second time. The decoder draws its own noise, so pass
# a generator for reproducible decoding.
video = decode_pipe(
    latents, generator=generator, output_type="np", denormalize=False, return_dict=False
)[0]

encode_video(
    video[0],
    fps=24,
    output_path="ltx25_diffusion_decode.mp4",
    audio=audio[0].float().cpu(),
    audio_sample_rate=pipe.vocoder.config.output_sampling_rate,
)

enable_tiling() bounds peak memory by the tile size rather than by the size of the video. The cheap early upsampling stages still see the full latent; only the last upsampling stage and the diffusion stage, which dominate decode memory, run per tile, so tiling changes the output only near tile borders. Each tile is denoised separately, so a tiled decode does not reproduce an untiled one exactly. The default tile and overlap sizes match the reference implementation's and can be overridden with the tile_sample_min_* / tile_sample_stride_* arguments.

Full / SFT transformer

transformer_full/ is not in model_index.json , so load it explicitly, and re-enable the shifting the distilled scheduler/ turns off. The pipeline's guidance defaults are already the SFT values, so the only other change is dropping the distilled overrides. As a diff against the quick start:

 import torch
-from diffusers import LTX2Pipeline
-from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES
+from diffusers import FlowMatchEulerDiscreteScheduler, LTX2Pipeline, LTX2VideoTransformer3DModel
+from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT
 from diffusers.utils import encode_video

 MODEL_ID = "Lightricks/LTX-2.5-Diffusers"

-pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16)
+pipe = LTX2Pipeline.from_pretrained(
+    MODEL_ID,
+    transformer=LTX2VideoTransformer3DModel.from_pretrained(
+        MODEL_ID, subfolder="transformer_full", dtype=torch.bfloat16
+    ),
+    dtype=torch.bfloat16,
+)
 pipe.enable_model_cpu_offload()
+pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
+    pipe.scheduler.config, use_dynamic_shifting=True, shift_terminal=0.1
+)

 video, audio = pipe(
     prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, "
            "the camera tracking alongside, snow crunching underfoot.",
     negative_prompt=DEFAULT_NEGATIVE_PROMPT,
     width=960,
     height=544,
     num_frames=121,
     frame_rate=24.0,
-    sigmas=DISTILLED_SIGMA_VALUES,
-    guidance_scale=1.0,
-    audio_guidance_scale=1.0,
-    stg_scale=0.0,
-    audio_stg_scale=0.0,
-    modality_scale=1.0,
-    audio_modality_scale=1.0,
     generator=torch.Generator("cuda").manual_seed(42),
     output_type="np",
     return_dict=False,
 )

Passing transformer= also keeps from_pretrained from fetching the distilled folder.

Prompting

The model was trained on long, single-paragraph audio-visual captions and degrades on short prompts. Describe the shot, the motion, the light and the sound in one paragraph, as the examples above do. Prompt enhancement for 2.5 uses a separate Gemma 4 checkpoint (e.g. google/gemma-4-E2B-it ), not this repo's fine-tuned text encoder.

Notes

• Distilled ( transformer/ ) is for few-step inference with an explicit sigmas= schedule and all guidance disabled; the full DiT ( transformer_full/ ) uses step counts and real guidance.
• The diffusion decoder denoises rather than decoding deterministically, so decoding is only reproducible with a generator .
• processor and prompt_enhancer appear in model_index.json but are not shipped here; they load as None .