thoughtworks / Qwen3-Coder-Next-Eagle3-exp-e

EAGLE3 Draft Model for Qwen3-Coder-Next

An EAGLE3 draft model that accelerates inference for Qwen3-Coder-Next (80B MoE, 512 experts, 10 active, GDN+attention hybrid) through speculative decoding.

1.37x mean speedup at B=1 across 4 benchmarks on 4x H200.

Results

Verified 2026-04-12 on 4x NVIDIA H200 144GB, TP=4, Triton, temp=0, max_tokens=512.

B=1 (single request)

B=32 (32 concurrent requests)

Protocol : B=1: 5 warmup + 20 measured (sequential). B=32: 15 warmup + 60 measured (32 concurrent). Metrics from server-side Prometheus.

Architecture

Training

54K samples (45% ShareGPT, 35% UltraChat, 20% PerfectBlend). 6 epochs, LR=1e-4, max_length=2048, batch_size=1, TP=4. Trained with --target-model-backend sglang --sglang-attention-backend triton via SpecForge .

Best training accuracy ( acc_0 ): 0.97. Note: training accuracy does not predict inference accept rate — there is a 30-60pp gap.

Usage

Benchmarked with our SGLang fork ( tails-mpt/sglang , commit 63291f7f51 ). Upstream SGLang may produce different speedups due to scheduling overhead differences.

python -m sglang.launch_server \
  --model-path <path-to-Qwen3-Coder-Next> \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path thoughtworks/Qwen3-Coder-Next-Eagle3-exp-e \
  --speculative-num-steps 3 \
  --speculative-num-draft-tokens 8 \
  --speculative-eagle-topk 4 \
  --attention-backend triton \
  --tp 4 --trust-remote-code --port 30000 \
  --enable-metrics --mem-fraction-static 0.65

Pinned dependencies : sgl-kernel 0.3.18.post2, flashinfer 0.6.6, torch 2.9.1+cu126.

Important : Use --attention-backend triton — FlashInfer is incompatible with Qwen3-Coder-Next's head_dim=256 hybrid layers.

Verify accept rate > 0% after startup to confirm the draft model loaded correctly.

Citation

@article{li2025eagle3,
  title={EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test},
  author={Li, Yuhui and Wei, Fangyun and Zhang, Chao and others},
  journal={arXiv preprint arXiv:2503.01840},
  year={2025}
}