ISTA-DASLab / Kimi-K2.5-2Bit-GSQ

huggingface.co
Total runs: 34
24-hour runs: 0
7-day runs: -2
30-day runs: -38
Model's Last Updated: June 18 2026
image-text-to-text

Introduction of Kimi-K2.5-2Bit-GSQ

Model Details of Kimi-K2.5-2Bit-GSQ

Kimi-K2.5 — 2-bit GSQ

2-bit quantization of moonshotai/Kimi-K2.5 (MoE, 384 experts, ≈260 GB FP) produced with GSQ (Gumbel-Softmax Quantization). The model is compressed from ≈4.5 bpp down to ≈2.13 bpp while preserving most of the base model's reasoning, coding, and long-context behaviour — and slightly exceeds the FP base on MATH 500 and LiveCodeBench v6 under our evaluation pipeline.

Quantization details
  • Base model: moonshotai/Kimi-K2.5
  • Bits / weight (effective): ≈2.13 bpp
  • Codebook: 2-bit symmetric scalar {-2, -1, 0, +1} × scale
  • Group size: 128
  • Format: compressed-tensors (auto-detected by vLLM)
  • Pipeline: GPTQ initialization → Gumbel-Softmax refinement (Lion optimizer)
  • Attention projections: kept in FP (only experts / MLPs quantized)
Serving with vLLM

Hopper (sm_90) or Ampere (sm ≥ 80) GPUs required for serving. On 8× H100/H200, valid TP sizes are 1, 2, 4, 8 (Marlin MoE constraint with group size 128).

vllm serve ISTA-DASLab/Kimi-K2.5-2Bit-GSQ \
  --tensor-parallel-size 8 \
  --trust-remote-code
Citation
@article{gsq2026,
  title  = {GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling},
  author = {Dadgarnia, Alireza and Tabesh, Soroush and Nikdan, Mahdi and Helcig, Michael and Kurti{\'c}, Eldar and Kleinegger, Max and Alistarh, Dan},
  journal= {arXiv preprint arXiv:2604.18556},
  year   = {2026},
  url    = {https://arxiv.org/abs/2604.18556}
}

Runs of ISTA-DASLab Kimi-K2.5-2Bit-GSQ on huggingface.co

34
Total runs
0
24-hour runs
2
3-day runs
-2
7-day runs
-38
30-day runs

More Information About Kimi-K2.5-2Bit-GSQ huggingface.co Model

More Kimi-K2.5-2Bit-GSQ license Visit here:

https://choosealicense.com/licenses/modified-mit

Kimi-K2.5-2Bit-GSQ huggingface.co

Kimi-K2.5-2Bit-GSQ huggingface.co is an AI model on huggingface.co that provides Kimi-K2.5-2Bit-GSQ's model effect (), which can be used instantly with this ISTA-DASLab Kimi-K2.5-2Bit-GSQ model. huggingface.co supports a free trial of the Kimi-K2.5-2Bit-GSQ model, and also provides paid use of the Kimi-K2.5-2Bit-GSQ. Support call Kimi-K2.5-2Bit-GSQ model through api, including Node.js, Python, http.

Kimi-K2.5-2Bit-GSQ huggingface.co Url

https://huggingface.co/ISTA-DASLab/Kimi-K2.5-2Bit-GSQ

ISTA-DASLab Kimi-K2.5-2Bit-GSQ online free

Kimi-K2.5-2Bit-GSQ huggingface.co is an online trial and call api platform, which integrates Kimi-K2.5-2Bit-GSQ's modeling effects, including api services, and provides a free online trial of Kimi-K2.5-2Bit-GSQ, you can try Kimi-K2.5-2Bit-GSQ online for free by clicking the link below.

ISTA-DASLab Kimi-K2.5-2Bit-GSQ online free url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Kimi-K2.5-2Bit-GSQ

Kimi-K2.5-2Bit-GSQ install

Kimi-K2.5-2Bit-GSQ is an open source model from GitHub that offers a free installation service, and any user can find Kimi-K2.5-2Bit-GSQ on GitHub to install. At the same time, huggingface.co provides the effect of Kimi-K2.5-2Bit-GSQ install, users can directly use Kimi-K2.5-2Bit-GSQ installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Kimi-K2.5-2Bit-GSQ install url in huggingface.co:

https://huggingface.co/ISTA-DASLab/Kimi-K2.5-2Bit-GSQ

Url of Kimi-K2.5-2Bit-GSQ

Kimi-K2.5-2Bit-GSQ huggingface.co Url

Provider of Kimi-K2.5-2Bit-GSQ huggingface.co

ISTA-DASLab
ORGANIZATIONS

Other API from ISTA-DASLab