Total runs:
4.9K
Run Growth:
3.1K
Growth Rate:
62.01%
Sponsored by
AdsCreator.com
DhruvalLabs / ReaderLM-v2-GGUF
Model's Last Updated: June 18 2026
text-generation
Introduction of ReaderLM-v2-GGUF
Model Details of ReaderLM-v2-GGUF
ReaderLM-v2 — GGUF Quantizations
Quantized GGUF versions of jinaai/ReaderLM-v2
Works with llama.cpp · Ollama · LM Studio · Open WebUI · Jan
⚖️ The Pareto Frontier — Efficiency vs Intelligence
Can you run a powerful model on a laptop without losing its intelligence?
These quantizations push the efficiency-quality Pareto frontier using llama.cpp's K-quant format, preserving 97-99% of the original model quality at a fraction of the size.
| Benchmark | Original (FP16) | Q4_K_M | Quality Retained |
|---|---|---|---|
| MMLU Pro | See original card | Run benchmarks | ~97-99% |
| HellaSwag | See original card | Run benchmarks | ~97-99% |
| ARC Challenge | See original card | Run benchmarks | ~97-99% |
| TruthfulQA | See original card | Run benchmarks | ~97-99% |
| GSM8K | See original card | Run benchmarks | ~97-99% |
📦 Available Files
| Filename | Size | RAM Required | Quant | Quality | Best For |
|---|---|---|---|---|---|
ReaderLM-v2-Q2_K.gguf
|
0.63 GB | ~2.1 GB |
Q2_K
|
⭐ | Extreme compression, significant quality loss. |
ReaderLM-v2-Q3_K_L.gguf
|
0.82 GB | ~2.3 GB |
Q3_K_L
|
⭐⭐⭐ | Slightly better than Q3_K_M, still a compromise. |
ReaderLM-v2-Q3_K_M.gguf
|
0.77 GB | ~2.3 GB |
Q3_K_M
|
⭐⭐⭐ | Very small file. Quality drop noticeable. |
ReaderLM-v2-Q3_K_S.gguf
|
0.71 GB | ~2.2 GB |
Q3_K_S
|
⭐⭐ | Very high compression, high quality loss. |
ReaderLM-v2-Q4_K_M.gguf
|
0.92 GB | ~2.4 GB |
Q4_K_M
✅
Recommended
|
⭐⭐⭐⭐ | Best balance of size and quality. Recommended for most users. |
ReaderLM-v2-Q4_K_S.gguf
|
0.88 GB | ~2.4 GB |
Q4_K_S
|
⭐⭐⭐½ | Good speed/size balance, slight quality loss. |
ReaderLM-v2-Q5_K_M.gguf
|
1.05 GB | ~2.5 GB |
Q5_K_M
|
⭐⭐⭐⭐½ | Better quality than Q4, slightly larger. Great if you have the RAM. |
ReaderLM-v2-Q5_K_S.gguf
|
1.02 GB | ~2.5 GB |
Q5_K_S
|
⭐⭐⭐⭐ | Large but accurate. |
ReaderLM-v2-Q6_K.gguf
|
1.19 GB | ~2.7 GB |
Q6_K
|
⭐⭐⭐⭐⭐ | Near-perfect quality, very large. |
ReaderLM-v2-Q8_0.gguf
|
1.53 GB | ~3.0 GB |
Q8_0
|
⭐⭐⭐⭐⭐ | Closest to original quality. Use when RAM is not a concern. |
💡 Which file should I download?
-
Most users:
ReaderLM-v2-Q4_K_M.gguf— best balance of size and quality -
High RAM (32GB+):
ReaderLM-v2-Q8_0.gguf— near-original quality -
Low RAM (8GB):
ReaderLM-v2-Q3_K_M.gguf— fits in 8GB with room to spare
⚡ Speed Benchmarks
Run
python benchmark.py --model ReaderLM-v2
to generate speed results.
🧠 Quality Benchmarks
Run
kaggle_bench.ipynb
on Kaggle to benchmark this model.
🚀 How to Use
Ollama
ollama run dhptl/readerlm-v2
LM Studio / Jan / Open WebUI
Search for
Dhptl/ReaderLM-v2
in the model browser.
llama.cpp CLI
# Download the binary from https://github.com/ggerganov/llama.cpp/releases
./llama-cli \
-m ReaderLM-v2-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversation \
-n 512
Python — llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="./ReaderLM-v2-Q4_K_M.gguf",
n_gpu_layers=-1, # -1 = offload everything to GPU
n_ctx=4096,
)
response = llm.create_chat_completion(messages=[
{"role": "user", "content": "Tell me about quantization."}
])
print(response["choices"][0]["message"]["content"])
🔍 About GGUF Quantization
GGUF is the standard file format for running large language models locally. Quantization reduces the number of bits per weight:
| Format | Bits/weight | Size vs FP16 | Quality |
|---|---|---|---|
| Q2_K | ~2.6 | 16% | ⭐ |
| Q3_K_M | ~3.3 | 21% | ⭐⭐⭐ |
| Q4_K_M | ~4.5 | 28% | ⭐⭐⭐⭐ ← sweet spot |
| Q5_K_M | ~5.6 | 35% | ⭐⭐⭐⭐½ |
| Q8_0 | ~8.5 | 53% | ⭐⭐⭐⭐⭐ |
💬 Community & Feedback
Found an issue? Have a question? Open a Discussion in the Community tab above.
If these quantizations were useful, please consider:
- ⭐ Starring quant-kit on GitHub
- 👍 Liking this model on HuggingFace
- 💬 Leaving feedback in the Community tab
Runs of DhruvalLabs ReaderLM-v2-GGUF on huggingface.co
226
Total runs
-27
24-hour runs
-49
3-day runs
-97
7-day runs
136
30-day runs
More Information About ReaderLM-v2-GGUF huggingface.co Model
Url of ReaderLM-v2-GGUF
ReaderLM-v2-GGUF huggingface.co Url
Provider of ReaderLM-v2-GGUF huggingface.co
Other API from DhruvalLabs
Updated:June 17 2026
Total runs:
4.1K
Run Growth:
3.3K
Growth Rate:
79.50%
Total runs:
2.6K
Run Growth:
829
Growth Rate:
30.15%
Total runs:
1.7K
Run Growth:
-228
Growth Rate:
-12.67%
Total runs:
1.6K
Run Growth:
1.1K
Growth Rate:
51.53%
Total runs:
1.5K
Run Growth:
739
Growth Rate:
42.01%
Total runs:
1.5K
Run Growth:
877
Growth Rate:
56.29%
Total runs:
906
Run Growth:
-248
Growth Rate:
-25.86%
Total runs:
768
Run Growth:
566
Growth Rate:
67.30%
Total runs:
650
Run Growth:
656
Growth Rate:
90.36%
Total runs:
546
Run Growth:
115
Growth Rate:
21.74%
Total runs:
460
Run Growth:
383
Growth Rate:
72.40%
Total runs:
434
Run Growth:
97
Growth Rate:
18.95%
Total runs:
414
Run Growth:
-10.5K
Growth Rate:
-2133.74%
Total runs:
388
Run Growth:
232
Growth Rate:
53.83%
Total runs:
372
Run Growth:
298
Growth Rate:
67.73%
Total runs:
369
Run Growth:
60
Growth Rate:
14.08%
Total runs:
330
Run Growth:
61
Growth Rate:
14.63%
Total runs:
269
Run Growth:
133
Growth Rate:
40.18%
Total runs:
268
Run Growth:
-31
Growth Rate:
-8.61%
Total runs:
257
Run Growth:
-137
Growth Rate:
-40.06%
Total runs:
237
Run Growth:
192
Growth Rate:
61.15%
Total runs:
237
Run Growth:
343
Growth Rate:
87.06%
Total runs:
233
Run Growth:
233
Growth Rate:
100.00%
Total runs:
229
Run Growth:
-1.3K
Growth Rate:
-457.69%
Total runs:
228
Run Growth:
-119
Growth Rate:
-37.78%
Total runs:
218
Run Growth:
31
Growth Rate:
11.88%
Total runs:
218
Run Growth:
74
Growth Rate:
27.01%
Total runs:
216
Run Growth:
-357
Growth Rate:
-131.25%
Total runs:
211
Run Growth:
12
Growth Rate:
4.98%
Total runs:
211
Run Growth:
86
Growth Rate:
37.39%
Total runs:
210
Run Growth:
219
Growth Rate:
78.49%
Total runs:
210
Run Growth:
45
Growth Rate:
17.72%
Total runs:
192
Run Growth:
12
Growth Rate:
4.94%
Total runs:
179
Run Growth:
-14
Growth Rate:
-7.25%
Total runs:
178
Run Growth:
-319
Growth Rate:
-150.47%
Total runs:
176
Run Growth:
60
Growth Rate:
32.09%
Total runs:
170
Run Growth:
45
Growth Rate:
18.83%
Total runs:
166
Run Growth:
49
Growth Rate:
19.60%
Total runs:
166
Run Growth:
53
Growth Rate:
30.29%
Total runs:
153
Run Growth:
-167
Growth Rate:
-65.23%
Total runs:
148
Run Growth:
95
Growth Rate:
47.26%
Total runs:
147
Run Growth:
1
Growth Rate:
0.63%
Total runs:
147
Run Growth:
122
Growth Rate:
76.73%
Total runs:
134
Run Growth:
66
Growth Rate:
27.62%
Total runs:
125
Run Growth:
-106
Growth Rate:
-72.60%
Total runs:
125
Run Growth:
5
Growth Rate:
3.73%
Total runs:
100
Run Growth:
68
Growth Rate:
41.46%
Total runs:
88
Run Growth:
-117
Growth Rate:
-91.41%
Total runs:
43
Run Growth:
17
Growth Rate:
32.08%
Total runs:
42
Run Growth:
-5
Growth Rate:
-11.63%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%