2-bit quantization of
moonshotai/Kimi-K2.5
(MoE, 384 experts, ≈260 GB FP) produced with
GSQ
(Gumbel-Softmax Quantization). The model is compressed from ≈4.5 bpp down to
≈2.13 bpp
while preserving most of the base model's reasoning, coding,
and long-context behaviour — and slightly
exceeds
the FP base on MATH 500
and LiveCodeBench v6 under our evaluation pipeline.
Attention projections:
kept in FP (only experts / MLPs quantized)
Serving with vLLM
Hopper (sm_90) or Ampere (sm ≥ 80) GPUs required for serving. On 8× H100/H200,
valid TP sizes are
1, 2, 4, 8
(Marlin MoE constraint with group size 128).
@article{gsq2026,
title = {GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling},
author = {Dadgarnia, Alireza and Tabesh, Soroush and Nikdan, Mahdi and Helcig, Michael and Kurti{\'c}, Eldar and Kleinegger, Max and Alistarh, Dan},
journal= {arXiv preprint arXiv:2604.18556},
year = {2026},
url = {https://arxiv.org/abs/2604.18556}
}
Runs of ISTA-DASLab Kimi-K2.5-2Bit-GSQ on huggingface.co
34
Total runs
0
24-hour runs
2
3-day runs
-2
7-day runs
-38
30-day runs
More Information About Kimi-K2.5-2Bit-GSQ huggingface.co Model
Kimi-K2.5-2Bit-GSQ huggingface.co is an AI model on huggingface.co that provides Kimi-K2.5-2Bit-GSQ's model effect (), which can be used instantly with this ISTA-DASLab Kimi-K2.5-2Bit-GSQ model. huggingface.co supports a free trial of the Kimi-K2.5-2Bit-GSQ model, and also provides paid use of the Kimi-K2.5-2Bit-GSQ. Support call Kimi-K2.5-2Bit-GSQ model through api, including Node.js, Python, http.
Kimi-K2.5-2Bit-GSQ huggingface.co is an online trial and call api platform, which integrates Kimi-K2.5-2Bit-GSQ's modeling effects, including api services, and provides a free online trial of Kimi-K2.5-2Bit-GSQ, you can try Kimi-K2.5-2Bit-GSQ online for free by clicking the link below.
ISTA-DASLab Kimi-K2.5-2Bit-GSQ online free url in huggingface.co:
Kimi-K2.5-2Bit-GSQ is an open source model from GitHub that offers a free installation service, and any user can find Kimi-K2.5-2Bit-GSQ on GitHub to install. At the same time, huggingface.co provides the effect of Kimi-K2.5-2Bit-GSQ install, users can directly use Kimi-K2.5-2Bit-GSQ installed effect in huggingface.co for debugging and trial. It also supports api for free installation.