DFlash speculator for the NVFP4 target
poolside/Laguna-S-2.1-NVFP4
. The
speculator is a 6-layer Laguna-style draft model (BF16); pair it with the NVFP4 base for
lower-latency serving via speculative decoding.
Trained:
e0630_rhiemann_baseline
SFT, DFlash Stage-2, 15k steps. Recommended
serving setting:
num_speculative_tokens=7
.
DFlash upstream support is in progress (vLLM #46853, SGLang #29446, TRT-LLM #15666). Use
poolside/Laguna-S-2.1-NVFP4
as the target model.
Benchmarks
Measured with TP=2,
temperature=0
, and
num_speculative_tokens=15
.
Throughput speedup
Concurrency
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
1
3.324x
2.893x
3.692x
2.426x
2.338x
4
2.498x
2.174x
2.742x
1.848x
1.831x
8
2.279x
1.948x
2.634x
1.719x
1.772x
16
2.302x
1.965x
2.626x
1.731x
1.672x
Acceptance length
Concurrency
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
1
5.775
4.942
6.438
4.171
4.017
4
5.804
4.935
6.298
4.161
4.091
8
5.719
4.885
6.537
4.193
4.373
16
5.758
4.896
6.412
4.123
3.981
Runs of poolside Laguna-S-2.1-DFlash-NVFP4 on huggingface.co
17.7K
Total runs
-618
24-hour runs
-1.2K
3-day runs
-2.7K
7-day runs
-28.9K
30-day runs
More Information About Laguna-S-2.1-DFlash-NVFP4 huggingface.co Model
Laguna-S-2.1-DFlash-NVFP4 huggingface.co
Laguna-S-2.1-DFlash-NVFP4 huggingface.co is an AI model on huggingface.co that provides Laguna-S-2.1-DFlash-NVFP4's model effect (), which can be used instantly with this poolside Laguna-S-2.1-DFlash-NVFP4 model. huggingface.co supports a free trial of the Laguna-S-2.1-DFlash-NVFP4 model, and also provides paid use of the Laguna-S-2.1-DFlash-NVFP4. Support call Laguna-S-2.1-DFlash-NVFP4 model through api, including Node.js, Python, http.
Laguna-S-2.1-DFlash-NVFP4 huggingface.co is an online trial and call api platform, which integrates Laguna-S-2.1-DFlash-NVFP4's modeling effects, including api services, and provides a free online trial of Laguna-S-2.1-DFlash-NVFP4, you can try Laguna-S-2.1-DFlash-NVFP4 online for free by clicking the link below.
poolside Laguna-S-2.1-DFlash-NVFP4 online free url in huggingface.co:
Laguna-S-2.1-DFlash-NVFP4 is an open source model from GitHub that offers a free installation service, and any user can find Laguna-S-2.1-DFlash-NVFP4 on GitHub to install. At the same time, huggingface.co provides the effect of Laguna-S-2.1-DFlash-NVFP4 install, users can directly use Laguna-S-2.1-DFlash-NVFP4 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Laguna-S-2.1-DFlash-NVFP4 install url in huggingface.co: