Sponsored by
Vmake Labs Video Enhancer
AngelSlim / Qwen3-8B-MTP-TTT3
Model's Last Updated: July 29 2026
Introduction of Qwen3-8B-MTP-TTT3
Model Details of Qwen3-8B-MTP-TTT3
A more accessible, comprehensive, and efficient toolkit for large model compression.
✒️
TechnicalReport
| 📖
Documentation
| 🤗
Hugging Face
| 🤖
ModelScope
Qwen3-8B No-Thinking Drafters: MTP and DFly
Model Summary
This repository contains speculative-decoding drafter checkpoints for Qwen3-8B , including an MTP drafter and a DFly drafter . The drafters propose candidate tokens for the target model to verify, improving inference throughput while preserving the target model's output distribution.
The models were trained with AngelSpec and are intended to be deployed in vLLM together with the Qwen3-8B target model.
- Target model: Qwen3-8B
- Inference mode: No-thinking
- Drafter types: MTP and DFly
- Training framework: AngelSpec
- Recommended serving backend: vLLM speculative decoding
- MTP speculative tokens: 3
- DFly speculative tokens: 8
Training Data
For Qwen3-8B, prompts are taken from Open-PerfectBlend (Xu et al., 2024). Responses are regenerated with the Qwen3-8B target model in non-thinking mode , and the resulting target-generated responses are used to train the drafter.
Regenerating responses with the same target model helps the drafter better match the target distribution, which is critical for speculative decoding acceptance rate and end-to-end acceleration.
Training Hyperparameters
These Qwen3-8B draft models use the following training configuration:
| Hyperparameter | Value |
|---|---|
| Batch size | 512 |
| Loss schedule | LK loss cold start for 300 steps, then end-to-end TV loss |
| Learning rate | 6.0e-4 |
| Minimum learning rate | 6.0e-5 |
| LR decay style | Cosine |
| Warmup ratio | 0.04 |
| Weight decay | 0.0 |
| Epochs | 10 |
| Maximum sequence length | 4096 |
Evaluation
The table reports acceleration-related metrics across math, code, and chat benchmarks.
| Target Model | Drafter | Math500 | GSM8K | HumanEval | MBPP | LiveCodeBench | MT-Bench | Avg. |
|---|---|---|---|---|---|---|---|---|
| Qwen3-8B | MTP | 3.53 | 3.56 | 3.33 | 3.22 | 3.25 | 2.57 | 3.24 |
| Qwen3-8B | DFlash | 4.97 | 5.54 | 4.77 | 4.50 | 4.46 | 3.16 | 4.57 |
| Qwen3-8B | DSpark | 5.87 | 6.25 | 5.56 | 5.25 | 5.20 | 3.77 | 5.32 |
| Qwen3-8B | DFly | 6.06 | 6.42 | 5.60 | 5.34 | 5.36 | 3.67 | 5.41 |
DFly achieves the highest average result among the listed Qwen3-8B drafters in this evaluation, with an average score of 5.41.
Deployment
Set
TARGET
to the Qwen3-8B target model and
DRAFT
to the corresponding drafter repository or local checkpoint path. The relevant inference code can be found in PR
https://github.com/vllm-project/vllm/pull/50246
.
MTP-3
export TARGET="Qwen/Qwen3-8B"
export DRAFT="<path-or-hf-repo-to-this-MTP-drafter>"
vllm serve "$TARGET" \
--tensor-parallel-size 1 \
--speculative-config "{"method":"mtp","model":"$DRAFT","num_speculative_tokens":3}"
DFly-7
export TARGET="Qwen/Qwen3-8B"
export DRAFT="<path-or-hf-repo-to-this-DFly-drafter>"
vllm serve "$TARGET" \
--tensor-parallel-size 1 \
--speculative-config "{"method":"dspark","model":"$DRAFT","num_speculative_tokens":7}"
Technical Report
For more details, see the technical report:
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
arXiv: https://arxiv.org/abs/2607.25852
Runs of AngelSlim Qwen3-8B-MTP-TTT3 on huggingface.co
42
Total runs
0
24-hour runs
0
3-day runs
21
7-day runs
21
30-day runs
More Information About Qwen3-8B-MTP-TTT3 huggingface.co Model
Url of Qwen3-8B-MTP-TTT3
Qwen3-8B-MTP-TTT3 huggingface.co Url
Provider of Qwen3-8B-MTP-TTT3 huggingface.co
Other API from AngelSlim
Total runs:
134.3K
Run Growth:
-199.6K
Growth Rate:
-156.66%
Total runs:
92.0K
Run Growth:
31.4K
Growth Rate:
34.13%
Total runs:
3.0K
Run Growth:
-1.7K
Growth Rate:
-57.26%
Total runs:
2.4K
Run Growth:
-813
Growth Rate:
-34.09%
Total runs:
1.6K
Run Growth:
251
Growth Rate:
16.25%
Total runs:
696
Run Growth:
-24
Growth Rate:
-3.45%
Total runs:
667
Run Growth:
404
Growth Rate:
60.57%
Total runs:
664
Run Growth:
-786
Growth Rate:
-116.44%
Total runs:
649
Run Growth:
516
Growth Rate:
79.51%
Total runs:
344
Run Growth:
-386
Growth Rate:
-114.88%
Total runs:
297
Run Growth:
-374
Growth Rate:
-118.73%
Total runs:
287
Run Growth:
192
Growth Rate:
66.90%
Total runs:
252
Run Growth:
-81
Growth Rate:
-32.14%
Total runs:
230
Run Growth:
-440
Growth Rate:
-191.30%
Total runs:
212
Run Growth:
92
Growth Rate:
43.40%
Total runs:
158
Run Growth:
-970
Growth Rate:
-613.92%
Total runs:
146
Run Growth:
-1.3K
Growth Rate:
-888.36%
Total runs:
132
Run Growth:
-309
Growth Rate:
-234.09%
Total runs:
125
Run Growth:
-238
Growth Rate:
-181.68%
Total runs:
121
Run Growth:
15
Growth Rate:
11.54%
Total runs:
94
Run Growth:
-69
Growth Rate:
-73.40%
Total runs:
92
Run Growth:
-31
Growth Rate:
-33.70%
Total runs:
87
Run Growth:
46
Growth Rate:
52.87%
Total runs:
85
Run Growth:
-21
Growth Rate:
-24.71%
Total runs:
80
Run Growth:
-7
Growth Rate:
-8.43%
Total runs:
77
Run Growth:
-98
Growth Rate:
-127.27%
Total runs:
74
Run Growth:
23
Growth Rate:
30.67%
Total runs:
72
Run Growth:
52
Growth Rate:
72.22%
Total runs:
58
Run Growth:
12
Growth Rate:
20.69%
Total runs:
57
Run Growth:
-95
Growth Rate:
-166.67%
Total runs:
52
Run Growth:
-4
Growth Rate:
-7.69%
Total runs:
40
Run Growth:
0
Growth Rate:
0.00%
Total runs:
37
Run Growth:
-21
Growth Rate:
-56.76%
Total runs:
36
Run Growth:
-9
Growth Rate:
-25.71%
Total runs:
36
Run Growth:
13
Growth Rate:
36.11%
Total runs:
36
Run Growth:
23
Growth Rate:
63.89%
Total runs:
35
Run Growth:
-10
Growth Rate:
-28.57%
Total runs:
34
Run Growth:
17
Growth Rate:
50.00%
Total runs:
34
Run Growth:
12
Growth Rate:
35.29%
Total runs:
33
Run Growth:
7
Growth Rate:
21.21%
Total runs:
33
Run Growth:
2
Growth Rate:
6.06%
Total runs:
33
Run Growth:
13
Growth Rate:
39.39%
Total runs:
31
Run Growth:
5
Growth Rate:
16.13%
Total runs:
31
Run Growth:
15
Growth Rate:
48.39%
Total runs:
31
Run Growth:
13
Growth Rate:
41.94%
Total runs:
30
Run Growth:
-2
Growth Rate:
-6.67%
Total runs:
30
Run Growth:
-245
Growth Rate:
-816.67%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
29
Run Growth:
-1
Growth Rate:
-3.45%
Total runs:
28
Run Growth:
17
Growth Rate:
60.71%
Total runs:
28
Run Growth:
2
Growth Rate:
6.90%
Total runs:
28
Run Growth:
16
Growth Rate:
57.14%
Total runs:
27
Run Growth:
-28
Growth Rate:
-103.70%
Total runs:
27
Run Growth:
5
Growth Rate:
18.52%
Total runs:
27
Run Growth:
0
Growth Rate:
0.00%
Total runs:
26
Run Growth:
9
Growth Rate:
34.62%
Total runs:
26
Run Growth:
9
Growth Rate:
34.62%
Total runs:
26
Run Growth:
6
Growth Rate:
23.08%
Total runs:
25
Run Growth:
-21
Growth Rate:
-84.00%
Total runs:
25
Run Growth:
14
Growth Rate:
56.00%
Total runs:
25
Run Growth:
14
Growth Rate:
56.00%
Total runs:
25
Run Growth:
7
Growth Rate:
28.00%
Total runs:
25
Run Growth:
0
Growth Rate:
0.00%
Total runs:
24
Run Growth:
11
Growth Rate:
45.83%
Total runs:
23
Run Growth:
-2
Growth Rate:
-8.70%
Total runs:
22
Run Growth:
-17
Growth Rate:
-77.27%
Total runs:
22
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
-1
Growth Rate:
-4.55%
Total runs:
21
Run Growth:
6
Growth Rate:
28.57%
Total runs:
21
Run Growth:
-12
Growth Rate:
-57.14%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
5
Growth Rate:
26.32%
Total runs:
19
Run Growth:
-6
Growth Rate:
-31.58%
Total runs:
19
Run Growth:
7
Growth Rate:
36.84%
Total runs:
19
Run Growth:
3
Growth Rate:
15.79%
Total runs:
19
Run Growth:
-4
Growth Rate:
-21.05%
Total runs:
18
Run Growth:
7
Growth Rate:
38.89%
Total runs:
18
Run Growth:
4
Growth Rate:
22.22%
Total runs:
18
Run Growth:
-6
Growth Rate:
-33.33%
Total runs:
18
Run Growth:
2
Growth Rate:
11.11%
Total runs:
18
Run Growth:
3
Growth Rate:
16.67%
Total runs:
18
Run Growth:
-39
Growth Rate:
-216.67%
Total runs:
17
Run Growth:
-11
Growth Rate:
-64.71%
Total runs:
17
Run Growth:
4
Growth Rate:
23.53%
Total runs:
17
Run Growth:
7
Growth Rate:
41.18%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
-12
Growth Rate:
-70.59%