amd / Kimi-K2-Thinking-MXFP4-AttnFP8

huggingface.co
Total runs: 96
24-hour runs: 1
7-day runs: -25
30-day runs: -491
Model's Last Updated: May 07 2026

Introduction of Kimi-K2-Thinking-MXFP4-AttnFP8

Model Details of Kimi-K2-Thinking-MXFP4-AttnFP8

Model Overview

  • Model Architecture: Kimi-K2-Thinking
    • Input: Text
    • Output: Text
  • Supported Hardware Microarchitecture: AMD MI350/MI355
  • ROCm: 7.0
  • Transformers: 4.57.6
  • Operating System(s): Linux
  • Inference Engine: vLLM
  • Model Optimizer: AMD-Quark (V0.11.2)
    • Quantized layers: experts , shared_experts , self_attn
    • Weight quantization: MoE OCP MXFP4, Static; self_attn Perchannel, FP8E4M3, Static
    • Activation quantization: MoE OCP MXFP4, Dynamic; self_attn Pertoken, FP8E4M3, Dynamic
  • Calibration Dataset: Pile

This model was built with Kimi-K2-Thinking model by applying AMD-Quark for MXFP4 quantization.

Model Quantization

The model was quantized from unsloth/Kimi-K2-Thinking-BF16 using AMD-Quark . Both weights and activations were quantized.

Quantization scripts:

cd Quark/examples/torch/language_modeling/llm_ptq/
exclude_layers="*mlp.gate *lm_head *mlp.gate_proj *mlp.up_proj *mlp.down_proj"

python quantize_quark.py \
    --model_dir unsloth/Kimi-K2-Thinking-BF16 \
    --quant_scheme mxfp4 \
    --layer_quant_scheme '*self_attn*' ptpc_fp8 \
    --exclude_layers  $exclude_layers \
    --output_dir  amd/Kimi-K2-Thinking-MXFP4-AttnFP8 \
    --file2file_quantization

Deployment

Use with vLLM

This model can be deployed efficiently using the vLLM backend.

Evaluation

The model was evaluated on GSM8K benchmarks.

Accuracy
Benchmark Kimi-K2-Thinking Kimi-K2-Thinking-MXFP4-AttnFP8(this model) Recovery
GSM8K (flexible-extract) 94.16 92.95 98.71%
Reproduction

The GSM8K results were obtained using the lm-evaluation-harness framework, based on the Docker image rocm/vllm-private:vllm_dev_base_mxfp4_20260122 , with vLLM, lm-eval and amd-quark compiled and installed from source inside the image.

Launching server
export VLLM_ATTENTION_BACKEND="TRITON_MLA"
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0

vllm serve amd/Kimi-K2-Thinking-MXFP4-AttnFP8 \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k2 \
  --reasoning-parser kimi_k2 \
  --trust-remote-code
Evaluating model in a new terminal
lm_eval \
  --model local-completions \
  --model_args "model=amd/Kimi-K2-Thinking-MXFP4-AttnFP8,base_url=http://0.0.0.0:8000/v1/completions,tokenized_requests=False,tokenizer_backend=None,num_concurrent=32" \
  --tasks gsm8k \
  --num_fewshot 5 \
  --batch_size 1

License

Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.

Runs of amd Kimi-K2-Thinking-MXFP4-AttnFP8 on huggingface.co

96
Total runs
1
24-hour runs
-8
3-day runs
-25
7-day runs
-491
30-day runs

More Information About Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co Model

More Kimi-K2-Thinking-MXFP4-AttnFP8 license Visit here:

https://choosealicense.com/licenses/modified-mit

Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co

Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co is an AI model on huggingface.co that provides Kimi-K2-Thinking-MXFP4-AttnFP8's model effect (), which can be used instantly with this amd Kimi-K2-Thinking-MXFP4-AttnFP8 model. huggingface.co supports a free trial of the Kimi-K2-Thinking-MXFP4-AttnFP8 model, and also provides paid use of the Kimi-K2-Thinking-MXFP4-AttnFP8. Support call Kimi-K2-Thinking-MXFP4-AttnFP8 model through api, including Node.js, Python, http.

Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co Url

https://huggingface.co/amd/Kimi-K2-Thinking-MXFP4-AttnFP8

amd Kimi-K2-Thinking-MXFP4-AttnFP8 online free

Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co is an online trial and call api platform, which integrates Kimi-K2-Thinking-MXFP4-AttnFP8's modeling effects, including api services, and provides a free online trial of Kimi-K2-Thinking-MXFP4-AttnFP8, you can try Kimi-K2-Thinking-MXFP4-AttnFP8 online for free by clicking the link below.

amd Kimi-K2-Thinking-MXFP4-AttnFP8 online free url in huggingface.co:

https://huggingface.co/amd/Kimi-K2-Thinking-MXFP4-AttnFP8

Kimi-K2-Thinking-MXFP4-AttnFP8 install

Kimi-K2-Thinking-MXFP4-AttnFP8 is an open source model from GitHub that offers a free installation service, and any user can find Kimi-K2-Thinking-MXFP4-AttnFP8 on GitHub to install. At the same time, huggingface.co provides the effect of Kimi-K2-Thinking-MXFP4-AttnFP8 install, users can directly use Kimi-K2-Thinking-MXFP4-AttnFP8 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Kimi-K2-Thinking-MXFP4-AttnFP8 install url in huggingface.co:

https://huggingface.co/amd/Kimi-K2-Thinking-MXFP4-AttnFP8

Url of Kimi-K2-Thinking-MXFP4-AttnFP8

Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co Url

Provider of Kimi-K2-Thinking-MXFP4-AttnFP8 huggingface.co

amd
ORGANIZATIONS

Other API from amd

huggingface.co

Total runs: 238.4K
Run Growth: 64.9K
Growth Rate: 26.61%
Updated:April 14 2026
huggingface.co

Total runs: 77.6K
Run Growth: -39.8K
Growth Rate: -51.24%
Updated:July 17 2026
huggingface.co

Total runs: 60.5K
Run Growth: 8.9K
Growth Rate: 14.56%
Updated:July 01 2026
huggingface.co

Total runs: 45.5K
Run Growth: -60.5K
Growth Rate: -133.28%
Updated:June 19 2026
huggingface.co

Total runs: 9.2K
Run Growth: -3.5K
Growth Rate: -39.81%
Updated:October 09 2024
huggingface.co

Total runs: 8.0K
Run Growth: -2.9K
Growth Rate: -34.59%
Updated:August 12 2025
huggingface.co

Total runs: 5.7K
Run Growth: -20.0K
Growth Rate: -349.28%
Updated:July 27 2026
huggingface.co

Total runs: 1.5K
Run Growth: 1.5K
Growth Rate: 97.70%
Updated:June 26 2025
huggingface.co

Total runs: 1.2K
Run Growth: -1.3K
Growth Rate: -108.67%
Updated:June 19 2026
huggingface.co

Total runs: 1.1K
Run Growth: 278
Growth Rate: 22.68%
Updated:May 27 2026
huggingface.co

Total runs: 1.1K
Run Growth: 672
Growth Rate: 60.70%
Updated:January 22 2026
huggingface.co

Total runs: 997
Run Growth: 747
Growth Rate: 71.28%
Updated:November 03 2025
huggingface.co

Total runs: 808
Run Growth: 256
Growth Rate: 32.08%
Updated:June 19 2026
huggingface.co

Total runs: 606
Run Growth: 471
Growth Rate: 77.85%
Updated:July 22 2026
huggingface.co

Total runs: 563
Run Growth: 363
Growth Rate: 64.94%
Updated:November 18 2025
huggingface.co

Total runs: 545
Run Growth: 27
Growth Rate: 4.96%
Updated:November 15 2025