Total runs:
11.0K
Run Growth:
9.3K
Growth Rate:
85.63%
Sponsored by
PostMcpAi
DhruvalLabs / Phi-4-mini-instruct-GGUF
Model's Last Updated: June 17 2026
text-generation
Introduction of Phi-4-mini-instruct-GGUF
Model Details of Phi-4-mini-instruct-GGUF
Phi-4-mini-instruct — GGUF Quantizations
Quantized GGUF versions of microsoft/Phi-4-mini-instruct
Works with llama.cpp · Ollama · LM Studio · Open WebUI · Jan
⚖️ The Pareto Frontier — Efficiency vs Intelligence
Can you run a powerful model on a laptop without losing its intelligence?
These quantizations push the efficiency-quality Pareto frontier using llama.cpp's K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
|---|---|---|---|
| MMLU Pro | See original card | Run benchmarks | ~97-99% |
| HellaSwag | See original card | Run benchmarks | ~97-99% |
| ARC Challenge | See original card | Run benchmarks | ~97-99% |
| TruthfulQA | See original card | Run benchmarks | ~97-99% |
| GSM8K | See original card | Run benchmarks | ~97-99% |
📦 Available Files
| Filename | Size | RAM Required | Quant | Quality | Best For |
|---|---|---|---|---|---|
Phi-4-mini-instruct-Q2_K.gguf
|
1.61 GB | ~3.1 GB |
Q2_K
|
⭐ | Extreme compression, significant quality loss. |
Phi-4-mini-instruct-Q3_K_L.gguf
|
2.15 GB | ~3.7 GB |
Q3_K_L
|
⭐⭐⭐ | Slightly better than Q3_K_M, still a compromise. |
Phi-4-mini-instruct-Q3_K_M.gguf
|
1.98 GB | ~3.5 GB |
Q3_K_M
|
⭐⭐⭐ | Very small file. Quality drop noticeable. |
Phi-4-mini-instruct-Q3_K_S.gguf
|
1.77 GB | ~3.3 GB |
Q3_K_S
|
⭐⭐ | Very high compression, high quality loss. |
Phi-4-mini-instruct-Q4_K_M.gguf
|
2.32 GB | ~3.8 GB |
Q4_K_M
✅
Recommended
|
⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
Phi-4-mini-instruct-Q4_K_S.gguf
|
2.18 GB | ~3.7 GB |
Q4_K_S
|
⭐⭐⭐½ | Good speed/size balance, slight quality loss. |
Phi-4-mini-instruct-Q5_K_M.gguf
|
2.62 GB | ~4.1 GB |
Q5_K_M
|
⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
Phi-4-mini-instruct-Q5_K_S.gguf
|
2.54 GB | ~4.0 GB |
Q5_K_S
|
⭐⭐⭐⭐ | Large but accurate. |
Phi-4-mini-instruct-Q6_K.gguf
|
2.94 GB | ~4.4 GB |
Q6_K
|
⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
Phi-4-mini-instruct-Q8_0.gguf
|
3.80 GB | ~5.3 GB |
Q8_0
|
⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
💡 Which file should I download?
-
Most users:
Phi-4-mini-instruct-Q4_K_M.gguf— best balance of size and quality -
High RAM (32GB+):
Phi-4-mini-instruct-Q8_0.gguf— near-original quality -
Low RAM (8GB):
Phi-4-mini-instruct-Q3_K_M.gguf— fits in 8GB with room to spare
⚡ Speed Benchmarks
Run
python benchmark.py --model Phi-4-mini-instruct
to generate speed results.
🧠 Quality Benchmarks
Run
kaggle_bench.ipynb
on Kaggle to benchmark this model.
🚀 How to Use
Ollama
ollama run dhptl/phi-4-mini-instruct
LM Studio / Jan / Open WebUI
Search for
Dhptl/Phi-4-mini-instruct
in the model browser.
llama.cpp CLI
# Download the binary from https://github.com/ggerganov/llama.cpp/releases
./llama-cli \
-m Phi-4-mini-instruct-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversation \
-n 512
Python — llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="./Phi-4-mini-instruct-Q4_K_M.gguf",
n_gpu_layers=-1, # -1 = offload everything to GPU
n_ctx=4096,
)
response = llm.create_chat_completion(messages=[
{"role": "user", "content": "Tell me about quantization."}
])
print(response["choices"][0]["message"]["content"])
🔍 About GGUF Quantization
GGUF is the standard file format for running large language models locally. Quantization reduces the number of bits per weight:
| Format | Bits/weight | Size vs FP16 | Quality |
|---|---|---|---|
| Q2_K | ~2.6 | 16% | ⭐ |
| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
💬 Community & Feedback
Found an issue? Have a question? Open a Discussion in the Community tab above.
If these quantizations were useful, please consider:
- ⭐ Starring quant-kit on GitHub
- 👍 Liking this model on HuggingFace
- 💬 Leaving feedback in the Community tab
Runs of DhruvalLabs Phi-4-mini-instruct-GGUF on huggingface.co
437
Total runs
121
24-hour runs
128
3-day runs
299
7-day runs
305
30-day runs
More Information About Phi-4-mini-instruct-GGUF huggingface.co Model
Url of Phi-4-mini-instruct-GGUF
Phi-4-mini-instruct-GGUF huggingface.co Url
Provider of Phi-4-mini-instruct-GGUF huggingface.co
Other API from DhruvalLabs
Updated:June 11 2026
Total runs:
2.1K
Run Growth:
-2.9K
Growth Rate:
-142.62%
Total runs:
2.0K
Run Growth:
-3.7K
Growth Rate:
-191.57%
Total runs:
1.9K
Run Growth:
1.1K
Growth Rate:
58.01%
Total runs:
1.6K
Run Growth:
1.6K
Growth Rate:
97.62%
Total runs:
1.2K
Run Growth:
803
Growth Rate:
66.53%
Total runs:
1.1K
Run Growth:
996
Growth Rate:
100.00%
Total runs:
1.0K
Run Growth:
726
Growth Rate:
71.18%
Total runs:
810
Run Growth:
-4.5K
Growth Rate:
-527.29%
Total runs:
686
Run Growth:
-153
Growth Rate:
-22.47%
Total runs:
632
Run Growth:
397
Growth Rate:
63.12%
Total runs:
538
Run Growth:
27
Growth Rate:
5.08%
Total runs:
470
Run Growth:
283
Growth Rate:
59.08%
Total runs:
425
Run Growth:
-354
Growth Rate:
-85.30%
Total runs:
412
Run Growth:
-1.4K
Growth Rate:
-348.85%
Total runs:
400
Run Growth:
329
Growth Rate:
79.47%
Total runs:
370
Run Growth:
-1.3K
Growth Rate:
-350.27%
Total runs:
369
Run Growth:
297
Growth Rate:
70.21%
Total runs:
364
Run Growth:
183
Growth Rate:
51.40%
Total runs:
294
Run Growth:
-1.7K
Growth Rate:
-617.45%
Total runs:
239
Run Growth:
118
Growth Rate:
48.16%
Total runs:
237
Run Growth:
-803
Growth Rate:
-349.13%
Total runs:
233
Run Growth:
233
Growth Rate:
100.00%
Total runs:
232
Run Growth:
19
Growth Rate:
7.54%
Total runs:
229
Run Growth:
123
Growth Rate:
53.71%
Total runs:
228
Run Growth:
-13
Growth Rate:
-5.63%
Total runs:
215
Run Growth:
200
Growth Rate:
100.00%
Total runs:
203
Run Growth:
-80
Growth Rate:
-39.80%
Total runs:
202
Run Growth:
95
Growth Rate:
45.89%
Total runs:
198
Run Growth:
-4
Growth Rate:
-1.91%
Total runs:
194
Run Growth:
70
Growth Rate:
36.08%
Total runs:
182
Run Growth:
-20
Growth Rate:
-11.56%
Total runs:
171
Run Growth:
51
Growth Rate:
32.28%
Total runs:
170
Run Growth:
36
Growth Rate:
18.18%
Total runs:
161
Run Growth:
94
Growth Rate:
66.20%
Total runs:
155
Run Growth:
-61
Growth Rate:
-42.96%
Total runs:
145
Run Growth:
15
Growth Rate:
10.42%
Total runs:
140
Run Growth:
-141
Growth Rate:
-96.58%
Total runs:
128
Run Growth:
17
Growth Rate:
13.18%
Total runs:
124
Run Growth:
-40
Growth Rate:
-32.79%
Total runs:
119
Run Growth:
45
Growth Rate:
35.43%
Total runs:
117
Run Growth:
-5
Growth Rate:
-4.10%
Total runs:
100
Run Growth:
-218
Growth Rate:
-227.08%
Total runs:
92
Run Growth:
-44
Growth Rate:
-55.00%
Total runs:
78
Run Growth:
35
Growth Rate:
33.02%
Total runs:
68
Run Growth:
-61
Growth Rate:
-87.14%
Total runs:
51
Run Growth:
-3
Growth Rate:
-8.11%
Total runs:
48
Run Growth:
-151
Growth Rate:
-314.58%
Total runs:
45
Run Growth:
-46
Growth Rate:
-90.20%
Total runs:
36
Run Growth:
-95
Growth Rate:
-263.89%
Total runs:
36
Run Growth:
-1.5K
Growth Rate:
-4188.89%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%