chutesai / Qwen3-32B-FP8-DFLASH

Qwen3-32B-FP8 DFlash Draft Model

DFlash speculative decoding draft model for Qwen/Qwen3-32B-FP8 . Trained using the DFlash (Block Diffusion for Flash Speculative Decoding) method from Z-Lab.

Architecture

The draft model takes concatenated hidden states from 5 target model layers as input and predicts a block of 16 tokens in parallel via iterative denoising. Attention is non-causal: queries attend to both target hidden states (context) and noise embeddings (draft tokens).

Training

Benchmarks

All benchmarks on a single NVIDIA RTX PRO 6000 Blackwell (98GB VRAM) using SGLang v0.5.13.post1.

ShareGPT (200 prompts, concurrency 8, max 1024 output tokens)

Synthetic (random tokens, 100 prompts per config)

Accept length increases with context length (2.13 at 128 tokens to 2.99 at 2048 tokens).

Usage (SGLang)

python -m sglang.launch_server \
    --model-path Qwen/Qwen3-32B-FP8 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path chutesai/Qwen3-32B-FP8-DFLASH \
    --speculative-num-draft-tokens 16 \
    --speculative-draft-attention-backend triton \
    --trust-remote-code \
    --mem-fraction-static 0.85 \
    --host 0.0.0.0 --port 30000

Requires SGLang >= v0.5.13 with DFlash support.