Myric / KAT-Coder-V2.5-Dev-MTP-head

KAT-Coder-V2.5-Dev — MTP head (experimental, un-fine-tuned baseline)

Kwaipilot/KAT-Coder-V2.5-Dev ships with mtp_num_hidden_layers: 0 — no MTP/NextN head at all, so no speculative-decoding speedup is possible out of the box. This repo adds one back by transplanting Qwen/Qwen3.6-35B-A3B 's own trained MTP head onto KAT-Coder. This is architecturally clean, not a hack: every relevant dimension (hidden_size, expert count, attention/MoE shapes) matches exactly between the two models, since KAT-Coder is itself built on the Qwen3.6-35B-A3B backbone.

This is the naive, un-fine-tuned transplant — the MTP head's weights are copied directly from stock Qwen3.6 with zero training on KAT-Coder's own hidden-state distribution. A fine-tuning pass (freezing KAT-Coder's own 40 layers entirely, training only this ~850M-parameter head) is a planned follow-up; this repo documents the honest baseline before that.

What's actually in this repo

Just one new shard ( model-00014-of-mtp.safetensors , 1.69GB) containing the 19 MTP-related tensors, plus a config.json / model.safetensors.index.json already merged to reference it alongside KAT-Coder's own 13 original shards. Nothing else was touched — KAT-Coder's own 13 shards are byte-identical to the original release; this is a drop-in addition, not a full re-upload of the 65GB checkpoint.

To use : put this repo's 3 files into a local copy of Kwaipilot/KAT-Coder-V2.5-Dev alongside its own 13 model-*-of-00013.safetensors shards (the config.json /index here already reference all 14 shards together).

Baseline: draft-acceptance rate (before fine-tuning)

Measured via llama-server --spec-type draft-mtp --spec-draft-n-max 2 on an APEX i-quality quant (4.67 bpw) built from this checkpoint, 5 real from-scratch coding prompts, n_predict=400 each:

For reference, the same KAT-Coder quant tier without any MTP head runs at ~70.8 tok/s solo decode speed (measured separately, no speculative decoding available at all). So even this un-fine-tuned transplant is already a real ~20-40% speedup, not a wash — acceptance rate varies by task in a way that tracks predictability (the BST task, mostly standard tree-traversal boilerplate, hits 89%; the LRU cache, with genuine locking/eviction logic, is the hardest at 61%).

Full coding-benchmark validation (real agentic tasks, not just completion speed)

Ran the same two task suites used elsewhere in this project's Qwen3.6-lineage work (an existing 5-task hard tier, plus a new 3-task tier designed to require genuine algorithmic reasoning rather than spec-transcription) against this MTP-enabled build vs the same quant without MTP, via opencode 's real agentic tool-calling loop (not just raw completion). Correctness was identical — 100% on both suites, both with and without MTP — exactly what the theory predicts: the MTP head only drafts candidate tokens, the main model always verifies against its own true computation, so a bad draft just gets rejected and never reaches the output. Speed differed substantially:

Both suites hit 100% (59/59 and 28/28 test cases respectively) in both configurations. The one slower result ( lru_ttl_cache , 0.81x) is most likely ordinary run-to-run variance in how much iteration that specific attempt needed, not a systematic MTP cost — every other task improved, and the hard-tier set as a whole is a genuine 2x.

Attribution

• KAT-Coder base model: Kwaipilot — Kwaipilot/KAT-Coder-V2.5-Dev (Apache-2.0).
• MTP head weights: Qwen — Qwen/Qwen3.6-35B-A3B (Apache-2.0), transplanted unmodified.
• Quantization/serving: llama.cpp ( ggml-org ).

Unofficial community experiment; not affiliated with or endorsed by Kwaipilot or Qwen.