ji-farthing / MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF

MiMo-V2.5-Pro DFlash Draft — BF16 GGUF (ik_llama.cpp)

A GGUF conversion of the DFlash block-diffusion draft for MiMo-V2.5-Pro , built for speculative decoding in ik_llama.cpp . This is the validation artifact for ik_llama.cpp PR #2048 (MiMo DFlash draft support).

⚠️ Not a standalone model. This is a 5-layer draft that shares the target's token embedding and output head. It only produces meaningful output when launched as --model-draft alongside a full MiMo-V2.5-Pro target GGUF. Loading it on its own will not generate coherent text.

Files

Matching target

Pair this draft with a MiMo-V2.5-Pro target GGUF. Validated against:

• gghfez/MiMo-V2.5-Pro-ik_llama-unfused-GGUF → MiMo-V2.5-Pro-IQ3_S-unfused.gguf (IQ3_S, ~349 GiB)

Use an unfused target GGUF. A fused-MoE MiMo GGUF produced garbled output with this draft in testing; the unfused tensor layout is required for the draft's captured-layer features to line up.

Usage (ik_llama.cpp)

llama-server \
  -m   MiMo-V2.5-Pro-IQ3_S-unfused.gguf \
  --model-draft mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf \
  -ngl 999 -ngld 999 \
  -sm graph --max-gpu 4 \
  -fa on -ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
  --spec-type dflash:n_max=1,p_min=0.0,cross_ctx=16 \
  -c 8192 --host 127.0.0.1 --port 8080

• --spec-type dflash:... selects the DFlash speculator. n_max=1 was optimal in testing (the draft runs autoregressively, so per-step overhead grows with n_max ). cross_ctx=16 is a small ring buffer of recent target positions visible to the draft; larger values collapsed acceptance.
• -sm graph --max-gpu 4 (tensor + graph split) was the best multi-GPU profile on 4×H200 NVL. Pinning the draft to one GPU with -devd added overhead and is not recommended under graph split.

Conversion metadata

Validation

4×H200 NVL (NVLink), ik_llama.cpp PR #2048. Draft accepted greedily at n_max=1, cross_ctx=16 , q4_0 KV on both target and draft.

Conversion notes

• Source draft weights: XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash ( dflash/ subdir, BF16 drafter).
• Vocab/tokenizer: shared from the MiMo-V2.5-Pro target (the draft carries no embeddings of its own).
• Converter: convert_hf_to_gguf.py on the filament/mimo-dflash-draft-convert branch (ik_llama.cpp PR #2048), DFlashDraftModel path, --outtype bf16 .

python convert_hf_to_gguf.py dflash \
  --target-model-dir . \
  --outtype bf16 \
  --outfile mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf