Total runs:
11.1K
Run Growth:
8.0K
Growth Rate:
71.59%
Sponsored by
iMean AI Trip Planner
DhruvalLabs / phi-4-GGUF
Model's Last Updated: June 16 2026
text-generation
Introduction of phi-4-GGUF
Model Details of phi-4-GGUF
phi-4 — GGUF Quantizations
Quantized GGUF versions of microsoft/phi-4
Works with llama.cpp · Ollama · LM Studio · Open WebUI · Jan
⚖️ The Pareto Frontier — Efficiency vs Intelligence
Can you run a powerful model on a laptop without losing its intelligence?
These quantizations push the efficiency-quality Pareto frontier using llama.cpp's K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
|---|---|---|---|
| MMLU Pro | See original card | Run benchmarks | ~97-99% |
| HellaSwag | See original card | Run benchmarks | ~97-99% |
| ARC Challenge | See original card | Run benchmarks | ~97-99% |
| TruthfulQA | See original card | Run benchmarks | ~97-99% |
| GSM8K | See original card | Run benchmarks | ~97-99% |
📦 Available Files
| Filename | Size | RAM Required | Quant | Quality | Best For |
|---|---|---|---|---|---|
phi-4-Q2_K.gguf
|
5.51 GB | ~7.0 GB |
Q2_K
|
⭐ | Extreme compression, significant quality loss. |
phi-4-Q3_K_L.gguf
|
7.57 GB | ~9.1 GB |
Q3_K_L
|
⭐⭐⭐ | Slightly better than Q3_K_M, still a compromise. |
phi-4-Q3_K_M.gguf
|
6.87 GB | ~8.4 GB |
Q3_K_M
|
⭐⭐⭐ | Very small file. Quality drop noticeable. |
phi-4-Q3_K_S.gguf
|
6.06 GB | ~7.6 GB |
Q3_K_S
|
⭐⭐ | Very high compression, high quality loss. |
phi-4-Q4_K_M.gguf
|
8.44 GB | ~9.9 GB |
Q4_K_M
✅
Recommended
|
⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
phi-4-Q4_K_S.gguf
|
7.88 GB | ~9.4 GB |
Q4_K_S
|
⭐⭐⭐½ | Good speed/size balance, slight quality loss. |
phi-4-Q5_K_M.gguf
|
9.78 GB | ~11.3 GB |
Q5_K_M
|
⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
phi-4-Q5_K_S.gguf
|
9.45 GB | ~11.0 GB |
Q5_K_S
|
⭐⭐⭐⭐ | Large but accurate. |
phi-4-Q6_K.gguf
|
11.20 GB | ~12.7 GB |
Q6_K
|
⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
phi-4-Q8_0.gguf
|
14.51 GB | ~16.0 GB |
Q8_0
|
⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
💡 Which file should I download?
-
Most users:
phi-4-Q4_K_M.gguf— best balance of size and quality -
High RAM (32GB+):
phi-4-Q8_0.gguf— near-original quality -
Low RAM (8GB):
phi-4-Q3_K_M.gguf— fits in 8GB with room to spare
⚡ Speed Benchmarks
Run
python benchmark.py --model phi-4
to generate speed results.
🧠 Quality Benchmarks
Run
kaggle_bench.ipynb
on Kaggle to benchmark this model.
🚀 How to Use
Ollama
ollama run dhptl/phi-4
LM Studio / Jan / Open WebUI
Search for
Dhptl/phi-4
in the model browser.
llama.cpp CLI
# Download the binary from https://github.com/ggerganov/llama.cpp/releases
./llama-cli \
-m phi-4-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversation \
-n 512
Python — llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="./phi-4-Q4_K_M.gguf",
n_gpu_layers=-1, # -1 = offload everything to GPU
n_ctx=4096,
)
response = llm.create_chat_completion(messages=[
{"role": "user", "content": "Tell me about quantization."}
])
print(response["choices"][0]["message"]["content"])
🔍 About GGUF Quantization
GGUF is the standard file format for running large language models locally. Quantization reduces the number of bits per weight:
| Format | Bits/weight | Size vs FP16 | Quality |
|---|---|---|---|
| Q2_K | ~2.6 | 16% | ⭐ |
| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
💬 Community & Feedback
Found an issue? Have a question? Open a Discussion in the Community tab above.
If these quantizations were useful, please consider:
- ⭐ Starring quant-kit on GitHub
- 👍 Liking this model on HuggingFace
- 💬 Leaving feedback in the Community tab
Runs of DhruvalLabs phi-4-GGUF on huggingface.co
181
Total runs
0
24-hour runs
145
3-day runs
144
7-day runs
50
30-day runs
More Information About phi-4-GGUF huggingface.co Model
Url of phi-4-GGUF
phi-4-GGUF huggingface.co Url
Provider of phi-4-GGUF huggingface.co
Other API from DhruvalLabs
Updated:June 11 2026
Total runs:
5.3K
Run Growth:
4.5K
Growth Rate:
83.49%
Total runs:
2.3K
Run Growth:
-4.2K
Growth Rate:
-181.36%
Total runs:
2.2K
Run Growth:
-2.9K
Growth Rate:
-129.71%
Total runs:
1.7K
Run Growth:
1.7K
Growth Rate:
98.64%
Total runs:
1.6K
Run Growth:
1.6K
Growth Rate:
100.00%
Total runs:
1.3K
Run Growth:
887
Growth Rate:
67.76%
Total runs:
1.3K
Run Growth:
927
Growth Rate:
72.48%
Total runs:
831
Run Growth:
-4.4K
Growth Rate:
-524.31%
Total runs:
812
Run Growth:
62
Growth Rate:
7.64%
Total runs:
686
Run Growth:
466
Growth Rate:
67.93%
Total runs:
633
Run Growth:
472
Growth Rate:
74.57%
Total runs:
573
Run Growth:
75
Growth Rate:
13.09%
Total runs:
564
Run Growth:
-1.2K
Growth Rate:
-218.44%
Total runs:
540
Run Growth:
330
Growth Rate:
61.11%
Total runs:
534
Run Growth:
387
Growth Rate:
72.47%
Total runs:
530
Run Growth:
341
Growth Rate:
64.34%
Total runs:
465
Run Growth:
-140
Growth Rate:
-30.11%
Total runs:
463
Run Growth:
164
Growth Rate:
35.42%
Total runs:
450
Run Growth:
-1.2K
Growth Rate:
-266.67%
Total runs:
409
Run Growth:
-1.5K
Growth Rate:
-372.13%
Total runs:
313
Run Growth:
313
Growth Rate:
100.00%
Total runs:
303
Run Growth:
124
Growth Rate:
40.92%
Total runs:
285
Run Growth:
119
Growth Rate:
41.75%
Total runs:
279
Run Growth:
60
Growth Rate:
21.51%
Total runs:
278
Run Growth:
-738
Growth Rate:
-265.47%
Total runs:
277
Run Growth:
176
Growth Rate:
63.54%
Total runs:
266
Run Growth:
175
Growth Rate:
65.79%
Total runs:
249
Run Growth:
61
Growth Rate:
24.50%
Total runs:
241
Run Growth:
-92
Growth Rate:
-38.17%
Total runs:
238
Run Growth:
182
Growth Rate:
76.47%
Total runs:
237
Run Growth:
130
Growth Rate:
54.85%
Total runs:
235
Run Growth:
81
Growth Rate:
34.47%
Total runs:
233
Run Growth:
233
Growth Rate:
100.00%
Total runs:
233
Run Growth:
-38
Growth Rate:
-16.31%
Total runs:
230
Run Growth:
56
Growth Rate:
24.35%
Total runs:
230
Run Growth:
123
Growth Rate:
53.48%
Total runs:
223
Run Growth:
47
Growth Rate:
21.08%
Total runs:
216
Run Growth:
176
Growth Rate:
81.48%
Total runs:
185
Run Growth:
85
Growth Rate:
45.95%
Total runs:
172
Run Growth:
-137
Growth Rate:
-79.65%
Total runs:
161
Run Growth:
39
Growth Rate:
24.22%
Total runs:
160
Run Growth:
84
Growth Rate:
52.50%
Total runs:
150
Run Growth:
76
Growth Rate:
50.67%
Total runs:
141
Run Growth:
32
Growth Rate:
22.70%
Total runs:
132
Run Growth:
2
Growth Rate:
1.52%
Total runs:
124
Run Growth:
56
Growth Rate:
45.16%
Total runs:
121
Run Growth:
25
Growth Rate:
20.66%
Total runs:
95
Run Growth:
-1.4K
Growth Rate:
-1503.16%
Total runs:
80
Run Growth:
39
Growth Rate:
48.75%
Total runs:
48
Run Growth:
-120
Growth Rate:
-250.00%
Total runs:
37
Run Growth:
-156
Growth Rate:
-421.62%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%