Total runs:
11.2K
Run Growth:
540
Growth Rate:
33.27%
Sponsored by
APIMart
DhruvalLabs / Kimi-Linear-48B-A3B-Instruct-GGUF
Model's Last Updated: June 25 2026
text-generation
Introduction of Kimi-Linear-48B-A3B-Instruct-GGUF
Model Details of Kimi-Linear-48B-A3B-Instruct-GGUF
Kimi-Linear-48B-A3B-Instruct — GGUF Quantizations
Quantized GGUF versions of moonshotai/Kimi-Linear-48B-A3B-Instruct
Works with llama.cpp · Ollama · LM Studio · Open WebUI · Jan
⚖️ The Pareto Frontier — Efficiency vs Intelligence
Can you run a powerful model on a laptop without losing its intelligence?
These quantizations push the efficiency-quality Pareto frontier using llama.cpp's K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
|---|---|---|---|
| MMLU Pro | See original card | Run benchmarks | ~97-99% |
| HellaSwag | See original card | Run benchmarks | ~97-99% |
| ARC Challenge | See original card | Run benchmarks | ~97-99% |
| TruthfulQA | See original card | Run benchmarks | ~97-99% |
| GSM8K | See original card | Run benchmarks | ~97-99% |
📦 Available Files
| Filename | Size | RAM Required | Quant | Quality | Best For |
|---|---|---|---|---|---|
Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf
|
27.66 GB | ~29.2 GB |
Q4_K_M
✅
Recommended
|
⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
Kimi-Linear-48B-A3B-Instruct-Q5_K_M.gguf
|
32.47 GB | ~34.0 GB |
Q5_K_M
|
⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
Kimi-Linear-48B-A3B-Instruct-Q5_K_S.gguf
|
31.56 GB | ~33.1 GB |
Q5_K_S
|
⭐⭐⭐⭐ | Large but accurate. |
Kimi-Linear-48B-A3B-Instruct-Q6_K.gguf
|
37.59 GB | ~39.1 GB |
Q6_K
|
⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
Kimi-Linear-48B-A3B-Instruct-Q8_0.gguf
|
48.66 GB | ~50.2 GB |
Q8_0
|
⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
💡 Which file should I download?
-
Most users:
Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf— best balance of size and quality -
High RAM (32GB+):
Kimi-Linear-48B-A3B-Instruct-Q8_0.gguf— near-original quality -
Low RAM (8GB):
Kimi-Linear-48B-A3B-Instruct-Q3_K_M.gguf— fits in 8GB with room to spare
⚡ Speed Benchmarks
Run
python benchmark.py --model Kimi-Linear-48B-A3B-Instruct
to generate speed results.
🧠 Quality Benchmarks
Run
kaggle_bench.ipynb
on Kaggle to benchmark this model.
🚀 How to Use
Ollama
ollama run dhptl/kimi-linear-48b-a3b-instruct
LM Studio / Jan / Open WebUI
Search for
Dhptl/Kimi-Linear-48B-A3B-Instruct
in the model browser.
llama.cpp CLI
# Download the binary from https://github.com/ggerganov/llama.cpp/releases
./llama-cli \
-m Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversation \
-n 512
Python — llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="./Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf",
n_gpu_layers=-1, # -1 = offload everything to GPU
n_ctx=4096,
)
response = llm.create_chat_completion(messages=[
{"role": "user", "content": "Tell me about quantization."}
])
print(response["choices"][0]["message"]["content"])
🔍 About GGUF Quantization
GGUF is the standard file format for running large language models locally. Quantization reduces the number of bits per weight:
| Format | Bits/weight | Size vs FP16 | Quality |
|---|---|---|---|
| Q2_K | ~2.6 | 16% | ⭐ |
| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
💬 Community & Feedback
Found an issue? Have a question? Open a Discussion in the Community tab above.
If these quantizations were useful, please consider:
- ⭐ Starring quant-kit on GitHub
- 👍 Liking this model on HuggingFace
- 💬 Leaving feedback in the Community tab
Runs of DhruvalLabs Kimi-Linear-48B-A3B-Instruct-GGUF on huggingface.co
4.2K
Total runs
-61
24-hour runs
-225
3-day runs
-262
7-day runs
3.3K
30-day runs
More Information About Kimi-Linear-48B-A3B-Instruct-GGUF huggingface.co Model
Url of Kimi-Linear-48B-A3B-Instruct-GGUF
Kimi-Linear-48B-A3B-Instruct-GGUF huggingface.co Url
Provider of Kimi-Linear-48B-A3B-Instruct-GGUF huggingface.co
Other API from DhruvalLabs
Updated:August 28 2026
Total runs:
4.9K
Run Growth:
3.0K
Growth Rate:
61.69%
Total runs:
2.6K
Run Growth:
538
Growth Rate:
20.89%
Total runs:
1.7K
Run Growth:
-375
Growth Rate:
-22.24%
Total runs:
1.5K
Run Growth:
784
Growth Rate:
53.33%
Total runs:
1.5K
Run Growth:
508
Growth Rate:
33.60%
Total runs:
905
Run Growth:
-301
Growth Rate:
-33.22%
Total runs:
766
Run Growth:
474
Growth Rate:
61.72%
Total runs:
658
Run Growth:
582
Growth Rate:
89.54%
Total runs:
449
Run Growth:
9
Growth Rate:
2.07%
Total runs:
436
Run Growth:
320
Growth Rate:
69.57%
Total runs:
425
Run Growth:
146
Growth Rate:
26.74%
Total runs:
421
Run Growth:
-10.6K
Growth Rate:
-2554.59%
Total runs:
389
Run Growth:
185
Growth Rate:
47.68%
Total runs:
370
Run Growth:
217
Growth Rate:
58.33%
Total runs:
362
Run Growth:
-1
Growth Rate:
-0.27%
Total runs:
268
Run Growth:
99
Growth Rate:
36.80%
Total runs:
261
Run Growth:
-144
Growth Rate:
-53.73%
Total runs:
258
Run Growth:
-59
Growth Rate:
-22.78%
Total runs:
256
Run Growth:
-213
Growth Rate:
-82.88%
Total runs:
255
Run Growth:
-34
Growth Rate:
-10.30%
Total runs:
235
Run Growth:
192
Growth Rate:
81.01%
Total runs:
233
Run Growth:
233
Growth Rate:
100.00%
Total runs:
226
Run Growth:
-19
Growth Rate:
-8.72%
Total runs:
223
Run Growth:
65
Growth Rate:
28.76%
Total runs:
218
Run Growth:
-1.4K
Growth Rate:
-599.13%
Total runs:
216
Run Growth:
-209
Growth Rate:
-91.67%
Total runs:
212
Run Growth:
-416
Growth Rate:
-192.59%
Total runs:
211
Run Growth:
-18
Growth Rate:
-8.53%
Total runs:
210
Run Growth:
151
Growth Rate:
71.90%
Total runs:
207
Run Growth:
113
Growth Rate:
47.68%
Total runs:
204
Run Growth:
66
Growth Rate:
31.28%
Total runs:
200
Run Growth:
12
Growth Rate:
5.71%
Total runs:
194
Run Growth:
-36
Growth Rate:
-18.75%
Total runs:
194
Run Growth:
3
Growth Rate:
1.38%
Total runs:
187
Run Growth:
-360
Growth Rate:
-202.25%
Total runs:
186
Run Growth:
-24
Growth Rate:
-14.12%
Total runs:
177
Run Growth:
-23
Growth Rate:
-12.85%
Total runs:
176
Run Growth:
57
Growth Rate:
32.39%
Total runs:
165
Run Growth:
49
Growth Rate:
29.52%
Total runs:
154
Run Growth:
-37
Growth Rate:
-22.29%
Total runs:
152
Run Growth:
-216
Growth Rate:
-141.18%
Total runs:
147
Run Growth:
70
Growth Rate:
47.30%
Total runs:
142
Run Growth:
-24
Growth Rate:
-16.33%
Total runs:
135
Run Growth:
96
Growth Rate:
65.31%
Total runs:
132
Run Growth:
-48
Growth Rate:
-35.82%
Total runs:
126
Run Growth:
-107
Growth Rate:
-85.60%
Total runs:
126
Run Growth:
-3
Growth Rate:
-2.40%
Total runs:
92
Run Growth:
-151
Growth Rate:
-171.59%
Total runs:
45
Run Growth:
-41
Growth Rate:
-80.39%
Total runs:
42
Run Growth:
7
Growth Rate:
16.28%
Total runs:
40
Run Growth:
-6
Growth Rate:
-14.29%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%