Total runs:
184.5K
Run Growth:
-13.6K
Growth Rate:
-7.35%
Sponsored by
YouArt
lmsys / SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge
Model's Last Updated: December 28 2025
Introduction of SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge
Model Details of SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge
EAGLE3 For meta-llama/Llama-3.1-8B-Instruct
About
SpecBundle is an open-source initiative, jointly driven by the community and industry, to democratize speculative decoding by providing high-performance speculative decoding draft weights for mainstream open-source models.
This checkpoint was trained by the SpecForge Team and released as the phase 1 of SpecBundle release . We regenerated the responses in the mlabonne/open-perfectblend and trained the model on 1.4M data samples for 2 epochs. This checkpoint was trained using the SpecForge framework.
Usage
You can use this checkpoint with the command below.
python3 -m sglang.launch_server \
--model meta-llama/Llama-3.1-8B-Instruct \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path lmsys/SGLang-EAGLE3-Llama-3.1-8B-Instruct-perfect-blend-regenerated \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4
Performance
This checkpoint exhibits superior performance on various benchmarks.
| Throughput | Acceptance Length |
|---|---|
|
|
You can reproduce the performance with the command below:
# clone specforge
git clone https://github.com/sgl-project/SpecForge.git
cd SpecForge/benchmarks
# run benchmarks
python bench_eagle3.py \
--model meta-llama/Llama-3.1-8B-Instruct \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path lmsys/SGLang-EAGLE3-Llama-3.3-70B-Instruct-perfect-blend-regenerated \
--port 30003 \
--config-list 1,3,1,4 1,5,1,6 1,5,3,6 1,7,1,8 1,7,4,8 \
--benchmark-list gsm8k math500 mtbench humaneval livecodebench financeqa gpqa \
--dtype bfloat16 \
--name llama3-perfect-blend-regen
Acknowledgement
We sincerely appreciate the collective efforts from both the developers in the open-source community and our industrial partners, especially Ant Group AQ Team, Meituan, Nex-AGI (Qiji Zhifeng), EigenAI for their invaluable contributions to the release of SpecBundle Phase 1.
Runs of lmsys SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge on huggingface.co
2.3K
Total runs
52
24-hour runs
-165
3-day runs
-38
7-day runs
637
30-day runs
More Information About SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge huggingface.co Model
Url of SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge
SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge huggingface.co Url
Provider of SGLang-EAGLE3-Llama-3.1-8B-Instruct-SpecForge huggingface.co
Other API from lmsys
Updated:March 13 2024
Total runs:
79.4K
Run Growth:
22.6K
Growth Rate:
28.52%
Total runs:
34.6K
Run Growth:
443
Growth Rate:
1.28%
Total runs:
34.0K
Run Growth:
-1.1K
Growth Rate:
-3.12%
Total runs:
25.9K
Run Growth:
24.9K
Growth Rate:
96.03%
Total runs:
18.3K
Run Growth:
-2.9K
Growth Rate:
-15.87%
Total runs:
15.4K
Run Growth:
-2.1K
Growth Rate:
-13.52%
Total runs:
11.9K
Run Growth:
-12.8K
Growth Rate:
-107.71%
Total runs:
7.8K
Run Growth:
5.0K
Growth Rate:
63.79%
Total runs:
5.2K
Run Growth:
-2.1K
Growth Rate:
-41.40%
Total runs:
3.3K
Run Growth:
-1.0K
Growth Rate:
-30.66%
Total runs:
3.3K
Run Growth:
306
Growth Rate:
9.32%
Total runs:
2.3K
Run Growth:
-2.5K
Growth Rate:
-111.40%
Total runs:
2.1K
Run Growth:
-16.6K
Growth Rate:
-799.61%
Total runs:
1.3K
Run Growth:
-1.2K
Growth Rate:
-87.65%
Total runs:
334
Run Growth:
236
Growth Rate:
70.66%
Total runs:
296
Run Growth:
-182
Growth Rate:
-61.49%
Total runs:
200
Run Growth:
10
Growth Rate:
5.00%
Total runs:
171
Run Growth:
64
Growth Rate:
37.43%
Total runs:
159
Run Growth:
-413
Growth Rate:
-259.75%
Total runs:
158
Run Growth:
66
Growth Rate:
41.77%
Total runs:
150
Run Growth:
-11
Growth Rate:
-7.33%
Total runs:
131
Run Growth:
-351
Growth Rate:
-267.94%
Total runs:
109
Run Growth:
92
Growth Rate:
84.40%
Total runs:
102
Run Growth:
-24
Growth Rate:
-23.53%
Total runs:
77
Run Growth:
58
Growth Rate:
75.32%
Total runs:
76
Run Growth:
56
Growth Rate:
73.68%
Total runs:
73
Run Growth:
-64
Growth Rate:
-87.67%
Total runs:
47
Run Growth:
47
Growth Rate:
100.00%
Total runs:
33
Run Growth:
33
Growth Rate:
100.00%
Total runs:
27
Run Growth:
27
Growth Rate:
100.00%
Total runs:
25
Run Growth:
25
Growth Rate:
100.00%
Total runs:
24
Run Growth:
11
Growth Rate:
45.83%
Total runs:
23
Run Growth:
7
Growth Rate:
30.43%
Total runs:
23
Run Growth:
6
Growth Rate:
26.09%
Total runs:
22
Run Growth:
-96
Growth Rate:
-436.36%
Total runs:
21
Run Growth:
21
Growth Rate:
100.00%
Total runs:
21
Run Growth:
21
Growth Rate:
100.00%
Total runs:
18
Run Growth:
18
Growth Rate:
100.00%
Total runs:
15
Run Growth:
3
Growth Rate:
20.00%
Total runs:
10
Run Growth:
-1
Growth Rate:
-10.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%