Total runs:
11.2K
Run Growth:
9.6K
Growth Rate:
85.53%
Sponsored by
Vmake Labs Video Enhancer
DhruvalLabs / VibeThinker-3B-GGUF
Model's Last Updated: June 29 2026
text-generation
Introduction of VibeThinker-3B-GGUF
Model Details of VibeThinker-3B-GGUF
VibeThinker-3B — GGUF Quantizations
Quantized GGUF versions of WeiboAI/VibeThinker-3B
Works with llama.cpp · Ollama · LM Studio · Open WebUI · Jan
Quantized by DhruvalLabs on June 29, 2026 using quant-kit
⚖️ The Pareto Frontier — Efficiency vs Intelligence
Can you run a powerful model on a laptop without losing its intelligence?
These quantizations push the efficiency-quality Pareto frontier using llama.cpp's K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
|---|---|---|---|
| MMLU Pro | See original card | Run benchmarks | ~97-99% |
| HellaSwag | See original card | Run benchmarks | ~97-99% |
| ARC Challenge | See original card | Run benchmarks | ~97-99% |
| TruthfulQA | See original card | Run benchmarks | ~97-99% |
| GSM8K | See original card | Run benchmarks | ~97-99% |
📦 Available Files
| Filename | Size | RAM Required | Quant | Quality | Best For |
|---|---|---|---|---|---|
VibeThinker-3B-Q2_K.gguf
|
1.19 GB | ~2.7 GB |
Q2_K
|
⭐ | Extreme compression, significant quality loss. |
VibeThinker-3B-Q3_K_L.gguf
|
1.59 GB | ~3.1 GB |
Q3_K_L
|
⭐⭐⭐ | Slightly better than Q3_K_M, still a compromise. |
VibeThinker-3B-Q3_K_M.gguf
|
1.48 GB | ~3.0 GB |
Q3_K_M
|
⭐⭐⭐ | Very small file. Quality drop noticeable. |
VibeThinker-3B-Q3_K_S.gguf
|
1.35 GB | ~2.9 GB |
Q3_K_S
|
⭐⭐ | Very high compression, high quality loss. |
VibeThinker-3B-Q4_K_M.gguf
|
1.80 GB | ~3.3 GB |
Q4_K_M
✅
Recommended
|
⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
VibeThinker-3B-Q4_K_S.gguf
|
1.71 GB | ~3.2 GB |
Q4_K_S
|
⭐⭐⭐½ | Good speed/size balance, slight quality loss. |
VibeThinker-3B-Q5_K_M.gguf
|
2.07 GB | ~3.6 GB |
Q5_K_M
|
⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
VibeThinker-3B-Q5_K_S.gguf
|
2.02 GB | ~3.5 GB |
Q5_K_S
|
⭐⭐⭐⭐ | Large but accurate. |
VibeThinker-3B-Q6_K.gguf
|
2.36 GB | ~3.9 GB |
Q6_K
|
⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
VibeThinker-3B-Q8_0.gguf
|
3.06 GB | ~4.6 GB |
Q8_0
|
⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
💡 Which file should I download?
-
Most users:
VibeThinker-3B-Q4_K_M.gguf— best balance of size and quality -
High RAM (32GB+):
VibeThinker-3B-Q8_0.gguf— near-original quality -
Low RAM (8GB):
VibeThinker-3B-Q3_K_M.gguf— fits in 8GB with room to spare
⚡ Speed Benchmarks
Run
python benchmark.py --model VibeThinker-3B
to generate speed results.
🧠 Quality Benchmarks
Run
kaggle_bench.ipynb
on Kaggle to benchmark this model.
🚀 How to Use
Ollama
ollama run dhruvallabs/vibethinker-3b
LM Studio / Jan / Open WebUI
Search for
DhruvalLabs/VibeThinker-3B
in the model browser.
llama.cpp CLI
# Download the binary from https://github.com/ggerganov/llama.cpp/releases
./llama-cli \
-m VibeThinker-3B-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversation \
-n 512
Python — llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="./VibeThinker-3B-Q4_K_M.gguf",
n_gpu_layers=-1, # -1 = offload everything to GPU
n_ctx=4096,
)
response = llm.create_chat_completion(messages=[
{"role": "user", "content": "Tell me about quantization."}
])
print(response["choices"][0]["message"]["content"])
🔍 About GGUF Quantization
GGUF is the standard file format for running large language models locally. Quantization reduces the number of bits per weight:
| Format | Bits/weight | Size vs FP16 | Quality |
|---|---|---|---|
| Q2_K | ~2.6 | 16% | ⭐ |
| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
💬 Community & Feedback
Found an issue? Have a question? Open a Discussion in the Community tab above.
If these quantizations were useful, please consider:
- ⭐ Starring quant-kit on GitHub
- 👍 Liking this model on HuggingFace
- 💬 Leaving feedback in the Community tab
Runs of DhruvalLabs VibeThinker-3B-GGUF on huggingface.co
372
Total runs
1
24-hour runs
0
3-day runs
-96
7-day runs
149
30-day runs
More Information About VibeThinker-3B-GGUF huggingface.co Model
Url of VibeThinker-3B-GGUF
VibeThinker-3B-GGUF huggingface.co Url
Provider of VibeThinker-3B-GGUF huggingface.co
Other API from DhruvalLabs
Updated:August 28 2026
Total runs:
4.2K
Run Growth:
3.3K
Growth Rate:
80.07%
Total runs:
3.8K
Run Growth:
1.5K
Growth Rate:
38.74%
Total runs:
2.4K
Run Growth:
260
Growth Rate:
10.46%
Total runs:
1.6K
Run Growth:
-3.7K
Growth Rate:
-231.63%
Total runs:
1.5K
Run Growth:
691
Growth Rate:
45.97%
Total runs:
1.5K
Run Growth:
198
Growth Rate:
13.41%
Total runs:
994
Run Growth:
-315
Growth Rate:
-31.69%
Total runs:
759
Run Growth:
350
Growth Rate:
46.11%
Total runs:
667
Run Growth:
517
Growth Rate:
77.51%
Total runs:
468
Run Growth:
-96
Growth Rate:
-20.51%
Total runs:
459
Run Growth:
226
Growth Rate:
49.24%
Total runs:
445
Run Growth:
-20
Growth Rate:
-4.49%
Total runs:
432
Run Growth:
-201
Growth Rate:
-46.53%
Total runs:
407
Run Growth:
-10.7K
Growth Rate:
-2634.15%
Total runs:
400
Run Growth:
201
Growth Rate:
50.25%
Total runs:
363
Run Growth:
-87
Growth Rate:
-23.97%
Total runs:
273
Run Growth:
43
Growth Rate:
15.75%
Total runs:
263
Run Growth:
-123
Growth Rate:
-46.77%
Total runs:
240
Run Growth:
-290
Growth Rate:
-120.83%
Total runs:
236
Run Growth:
107
Growth Rate:
45.34%
Total runs:
236
Run Growth:
-2
Growth Rate:
-0.85%
Total runs:
233
Run Growth:
233
Growth Rate:
100.00%
Total runs:
231
Run Growth:
-309
Growth Rate:
-133.77%
Total runs:
229
Run Growth:
-49
Growth Rate:
-21.40%
Total runs:
219
Run Growth:
-18
Growth Rate:
-8.22%
Total runs:
217
Run Growth:
-18
Growth Rate:
-8.29%
Total runs:
212
Run Growth:
-474
Growth Rate:
-223.58%
Total runs:
209
Run Growth:
48
Growth Rate:
22.97%
Total runs:
207
Run Growth:
-327
Growth Rate:
-157.97%
Total runs:
205
Run Growth:
-44
Growth Rate:
-21.46%
Total runs:
204
Run Growth:
-62
Growth Rate:
-30.39%
Total runs:
199
Run Growth:
-1.5K
Growth Rate:
-747.24%
Total runs:
195
Run Growth:
-118
Growth Rate:
-60.51%
Total runs:
192
Run Growth:
68
Growth Rate:
35.42%
Total runs:
187
Run Growth:
-92
Growth Rate:
-49.20%
Total runs:
187
Run Growth:
-43
Growth Rate:
-22.99%
Total runs:
185
Run Growth:
-388
Growth Rate:
-209.73%
Total runs:
181
Run Growth:
-122
Growth Rate:
-67.40%
Total runs:
162
Run Growth:
30
Growth Rate:
18.52%
Total runs:
154
Run Growth:
13
Growth Rate:
8.44%
Total runs:
149
Run Growth:
-314
Growth Rate:
-210.74%
Total runs:
144
Run Growth:
-41
Growth Rate:
-28.47%
Total runs:
144
Run Growth:
-97
Growth Rate:
-67.36%
Total runs:
133
Run Growth:
-152
Growth Rate:
-114.29%
Total runs:
133
Run Growth:
53
Growth Rate:
39.85%
Total runs:
128
Run Growth:
-32
Growth Rate:
-25.00%
Total runs:
89
Run Growth:
-83
Growth Rate:
-93.26%
Total runs:
85
Run Growth:
-192
Growth Rate:
-225.88%
Total runs:
47
Run Growth:
-2
Growth Rate:
-4.35%
Total runs:
45
Run Growth:
-76
Growth Rate:
-168.89%
Total runs:
43
Run Growth:
6
Growth Rate:
13.95%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%