thoughtworks / MiniMax-M2.5-Eagle3

EAGLE3 Draft Head — MiniMax-M2.5

A lightweight EAGLE3 draft head for MiniMax-M2.5 (229B MoE, ~10B active parameters). Trained with SpecForge on 8x H200 GPUs using the EAGLE-3 training-time test objective.

Blog post : 2x Faster on a 229B MoE: EAGLE3 Speculative Decoding for MiniMax-M2.5

Usage

SGLang (GPU)

Requires our SGLang fork for MiniMax-M2.5 Eagle3 support + FP8 dtype fixes.

B=1 server (wide tree — optimal for single-user, real-time requests):

pip install git+https://github.com/tails-mpt/sglang.git

python -m sglang.launch_server \
    --model-path MiniMaxAI/MiniMax-M2.5 \
    --speculative-algorithm EAGLE3 \
    --speculative-draft-model-path thoughtworks/MiniMax-M2.5-Eagle3 \
    --speculative-num-steps 3 \
    --speculative-num-draft-tokens 8 \
    --speculative-eagle-topk 4 \
    --dtype fp8 \
    --tp 4 \
    --port 30000

B=32 server (narrow tree — optimal for batch workloads):

python -m sglang.launch_server \
    --model-path MiniMaxAI/MiniMax-M2.5 \
    --speculative-algorithm EAGLE3 \
    --speculative-draft-model-path thoughtworks/MiniMax-M2.5-Eagle3 \
    --speculative-num-steps 5 \
    --speculative-num-draft-tokens 6 \
    --speculative-eagle-topk 1 \
    --dtype fp8 \
    --tp 4 \
    --port 30002

Important : Use different speculative configs for B=1 vs B=32. A wider tree (topk=4) exploits idle GPU compute at low batch; a narrow tree (topk=1) minimizes MoE expert dispatch overhead at high batch.

Python Client

import requests

response = requests.post(
    "http://localhost:30000/v1/chat/completions",
    json={
        "model": "default",
        "messages": [{"role": "user", "content": "Write a Python function to merge two sorted lists."}],
        "max_tokens": 512,
        "temperature": 0,
    }
)
print(response.json()["choices"][0]["message"]["content"])

Training Details

Training Method

EAGLE3 trains a single-layer draft head that predicts the next token using hidden states captured from three auxiliary layers of the target model (layers 1, 30, 58 — early, middle, and late). The training objective is the Training-Time Test (TTT) loss, which simulates the speculative decoding accept/reject process during training to maximize the expected number of accepted tokens at inference time.

Performance

Training Accuracy (base checkpoint, before regenerated data fine-tuning)

The released model was fine-tuned for 6 additional epochs on 20K regenerated samples from the target model. The fine-tuned accuracy is expected to be equal or higher than these base values.

Inference Benchmarks (B=1, temp=0, FP8, TP=4)

Config: steps=3, topk=4, draft_tokens=8. All datasets at temp=0 on 8x H200 (TP=4).

Model Architecture

Limitations

• TP=4 only. TP=8 fails due to FP8 block size constraint ( intermediate_size / 8 = 192 , not divisible by block_n=128 ).
• Temperature sensitivity. Best performance at temp=0 (greedy). At temp=0.7, B=1 speedup drops to 1.27-1.80x and some B=32 datasets regress below baseline.
• Coding-focused benchmarks. All benchmarks use coding-oriented datasets (HumanEval, SWEBench, Aider). Conversational workloads may show different patterns.
• SPEC_V2 incompatible. The overlap scheduler ( SGLANG_ENABLE_SPEC_V2=true ) is not supported — standard (non-overlapped) speculation only.
• Requires SGLang fork. Upstream SGLang does not yet include the FP8 dtype patches needed for Eagle3 on this model.

License

This draft head is released under Apache 2.0, matching the MiniMax-M2.5 license .

Citation

@inproceedings{li2025eagle3,
  title={{EAGLE-3}: Scaling up Inference Acceleration of Large Language Models via Training-Time Test},
  author={Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2025}
}