DFlash speculator for the FP8 target
poolside/Laguna-S-2.1-FP8
. The
speculator is a 6-layer Laguna-style draft model (BF16); pair it with the FP8 base for
lower-latency serving via speculative decoding.
Trained:
e0630_rhiemann_baseline
SFT, DFlash Stage-2, 15k steps. Recommended
serving setting:
num_speculative_tokens=7
.
DFlash upstream support is in progress (vLLM #46853, SGLang #29446, TRT-LLM #15666). Use
poolside/Laguna-S-2.1-FP8
as the target model.
Benchmarks
Measured with TP=2,
temperature=0
, and
num_speculative_tokens=15
.
Throughput speedup
Concurrency
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
1
3.179x
2.938x
3.269x
2.380x
2.603x
4
2.614x
2.423x
2.691x
1.963x
2.090x
8
2.666x
2.410x
2.803x
1.962x
2.230x
16
2.618x
2.364x
2.866x
2.031x
2.302x
Acceptance length
Concurrency
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
1
5.748
5.197
5.889
4.247
4.663
4
5.765
5.212
5.882
4.218
4.411
8
5.863
5.199
6.094
4.178
4.572
16
5.787
5.161
6.144
4.291
4.600
Runs of poolside Laguna-S-2.1-DFlash-FP8 on huggingface.co
1.0K
Total runs
-114
24-hour runs
-553
3-day runs
-565
7-day runs
-755
30-day runs
More Information About Laguna-S-2.1-DFlash-FP8 huggingface.co Model
Laguna-S-2.1-DFlash-FP8 huggingface.co
Laguna-S-2.1-DFlash-FP8 huggingface.co is an AI model on huggingface.co that provides Laguna-S-2.1-DFlash-FP8's model effect (), which can be used instantly with this poolside Laguna-S-2.1-DFlash-FP8 model. huggingface.co supports a free trial of the Laguna-S-2.1-DFlash-FP8 model, and also provides paid use of the Laguna-S-2.1-DFlash-FP8. Support call Laguna-S-2.1-DFlash-FP8 model through api, including Node.js, Python, http.
Laguna-S-2.1-DFlash-FP8 huggingface.co is an online trial and call api platform, which integrates Laguna-S-2.1-DFlash-FP8's modeling effects, including api services, and provides a free online trial of Laguna-S-2.1-DFlash-FP8, you can try Laguna-S-2.1-DFlash-FP8 online for free by clicking the link below.
poolside Laguna-S-2.1-DFlash-FP8 online free url in huggingface.co:
Laguna-S-2.1-DFlash-FP8 is an open source model from GitHub that offers a free installation service, and any user can find Laguna-S-2.1-DFlash-FP8 on GitHub to install. At the same time, huggingface.co provides the effect of Laguna-S-2.1-DFlash-FP8 install, users can directly use Laguna-S-2.1-DFlash-FP8 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Laguna-S-2.1-DFlash-FP8 install url in huggingface.co: