Abiray / KAT-Coder-V2.5-Dev-Imatrix-GGUF

KAT-Coder-V2.5-Dev (GGUF Quantizations)

This repository contains Importance-Matrix (imatrix) quantized GGUF files for Kwaipilot/KAT-Coder-V2.5-Dev .

KAT-Coder-V2.5-Dev is an open-weight, post-trained Mixture-of-Experts (MoE) coding agent model featuring 35B total parameters with 3B activated parameters per token , fine-tuned on top of Qwen3.6-35B-A3B .

⚠️ Note: This open-weight release contains only language-model weights and operates as a text-only model. Vision/multimodal components are not included.

📦 Provided GGUF Files

All quantizations in this repository were converted using llama.cpp and optimized using an imatrix (Importance Matrix) calibration file to maintain high performance at lower precision levels.

🚀 Quickstart & Usage

Running with llama.cpp

Ensure you are using a recent build of llama.cpp that supports Qwen3 / MoE architectures.

CLI Example:

./llama-cli -m KAT-Coder-V2.5-Dev-Q4_K_M.gguf \
    -p "Write a Python function that implements a binary search tree with deletion." \
    -n 4096 \
    -c 32768 \
    --temp 0.7

Launching an OpenAI-Compatible API Server:

./llama-server -m KAT-Coder-V2.5-Dev-Q4_K_M.gguf \
    --host 0.0.0.0 \
    --port 8000 \
    -c 262144 \
    -ngl 99

GUI Frontends (LM Studio, KoboldCpp, Jan)

1. Download your preferred .gguf file from the table above.
2. Place the file inside your local model folder (e.g., ~/.cache/lm-studio/models or KoboldCpp directory).
3. Set your context size up to 262,144 tokens (adjust depending on your available system RAM/VRAM).

✨ Original Model Highlights

• SOTA Agentic Coding Performance: Through post-training SFT and RL, KAT-Coder-V2.5-Dev achieves state-of-the-art results among models of similar parameter scales on benchmark tasks like SWE-bench Verified (69.40%) .
• Reduced Pathological Behaviors: Reinforcement Learning significantly reduced unwanted behaviors, such as abnormal tool labels (-9pp improvement) and single-turn continuous repetitions (reduced to 0%).
• Preserve Thinking Mode: The model supports retaining historical thinking context across multi-turn interactions, improving agent consistency and saving redundant reasoning tokens.

📊 Benchmark Performance

The table below shows the official benchmark evaluation results reproduced in-house by the original authors:

🔬 Post-Training Details

KAT-Coder-V2.5-Dev follows a two-stage post-training pipeline built on top of Qwen3.6-35B-A3B :

1. Supervised Fine-Tuning (SFT): Fine-tuned on 127K curated agentic and coding examples.
2. Reinforcement Learning (RL):
   • Token-in-Token-out (TITO) Consistency: Eliminates off-policy training discrepancies caused by tokenizer or chat-template changes.
   • Truncated Importance Sampling (TIS): Mitigates policy staleness during asynchronous rollout collection.
   • Reliable Execution Feedback: Built using verified sandbox execution for dense and reliable reward signals.
   • Specific Penalties: Introduced targeted reward penalties against abnormal parallel tool calling (70+ tool calls in one turn), failed calls, and loops.

📜 Citation

If you use KAT-Coder-V2.5-Dev or these GGUF quantizations in your work, please cite the technical report:

@misc{katcoder_v25_2026,
  title={{KAT-Coder-V2.5 Technical Report}},
  author={{KwaiKAT Team}},
  year={2026},
  month={July},
  eprint={2607.05471},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={[https://arxiv.org/pdf/2607.05471](https://arxiv.org/pdf/2607.05471)}
}

Original model created by the KwaiKAT Team / Kwaipilot . Quantized to GGUF format by Abiray .