multimodalart / laguna-s-2.1-DFlash

Laguna S 2.1 DFlash drafter — fixed GGUF (BF16 + Q8_0)

Metadata-corrected GGUF exports of the Laguna S 2.1 DFlash speculative-decoding drafter for llama.cpp ( poolside fork, branch laguna ).

Why this repo exists

The drafter GGUF shipped in poolside/Laguna-S-2.1-GGUF ( laguna-s-2.1-DFlash-BF16.gguf ) is missing two metadata keys relative to the checkpoint's HF config, which makes draft acceptance collapse with context length (27% at 767 tokens → 1.2% at 7K → 0.5% at 33K), turning DFlash into a net slowdown on long prompts:

These files have both keys corrected (plus the per-layer SWA pattern), restoring flat acceptance at any context length (~2.5–2.8 committed tokens/round, consistent with the model card's 2.9–3.1). vLLM is unaffected (it reads the HF config directly).

• laguna-s-2.1-DFlash-BF16-fixed.gguf — original weights, metadata fixed (2.2 GB)
• laguna-s-2.1-DFlash-Q8_0-fixed.gguf — Q8_0 quantized (1.1 GB, recommended). Drafter quantization cannot affect output quality — drafts are verified by the target model — and measured acceptance is unchanged, while halving the drafter's per-round weight reads.

Usage (Apple Silicon recipe)

Verify batches on this fine-grained MoE are expensive on memory-bound hardware, so shallow adaptive drafting wins. Measured on an M4 Max 128GB at ~10K context: ~1.25× over base decoding (44 vs 36.5 tok/s, greedy):

./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \
  -md laguna-s-2.1-DFlash-Q8_0-fixed.gguf \
  --spec-type draft-dflash --spec-draft-n-max 2 --spec-draft-p-min 0.5 \
  -fa on --jinja -c 262144 -ub 4096 --cache-reuse 256 --port 8000

On CUDA-class hardware, deeper drafting pays off — use the model card's --spec-draft-n-max 15 instead.

Related fixes in llama.cpp (branch apolinario/llama.cpp@laguna , partially upstreamed via poolsideai/llama.cpp#2 ): Metal f16-overflow sanitization of DFlash target features, SWA-pattern default for drafter GGUFs missing the key, and two Metal decode optimizations (~+10%).