Total runs:
734.4K
Run Growth:
603.3K
Growth Rate:
80.92%
Sponsored by
APIMart
AtomicChat / Qwen3.5-4B-GGUF
Model's Last Updated: July 23 2026
text-generation
Introduction of Qwen3.5-4B-GGUF
Model Details of Qwen3.5-4B-GGUF
Qwen3.5 4B , self-quantized to GGUF by Atomic Chat . Built straight from Qwen's original weights with a per-tensor importance matrix. Runs fully offline.
Highlights
- Efficient hybrid architecture combining Gated Delta Networks with sparse Mixture-of-Experts for high-throughput inference at low latency and cost.
- Unified vision-language foundation trained with early fusion on multimodal tokens (these GGUF quants cover the text path).
- 262,144-token native context , extensible up to ~1,010,000 tokens.
- Global linguistic coverage : Qwen reports support for 201 languages and dialects.
- Thinking and instruct modes , each with its own recommended sampling presets.
-
Full quant ladder
with an importance matrix on every quant over
calibration_datav3.
These GGUFs are self-quantized from the original weights , not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
Always pass
--jinjaso the Qwen3.5 4B chat template is applied. Without it the model can emit malformed turns.
Model Overview
| Property | Value |
|---|---|
| Base model |
Qwen/Qwen3.5-4B
|
| Total parameters | 4B |
| Layers | 32 |
| Context length | 262,144 native, extensible up to ~1,010,000 |
| Architecture | Causal LM with vision encoder; hybrid Gated DeltaNet + Gated Attention, hidden dim 2560, trained with MTP |
| This repo | GGUF quants (imatrix), text path |
Scores are Qwen's published results for the base
Qwen/Qwen3.5-4B
. Quantization preserves the large majority of this;
Q4_K_M
and up sit within a point or two of full precision.
Choosing a quant
| Quant | Size | Notes |
|---|---|---|
Q2_K
|
1.9 GB | Smallest. Minimal RAM, clear quality drop. |
IQ3_M
|
2.2 GB | Beats Q3 at similar size thanks to imatrix. Best low-RAM pick. |
Q3_K_M
|
2.3 GB | Low quality but usable. |
Q3_K_L
|
2.4 GB | A step above Q3_K_M. |
IQ4_XS
|
2.5 GB | Excellent quality for size. Recommended low-bit. |
Q4_K_S
|
2.6 GB | Compact Q4, fast. |
Q4_K_M
|
2.7 GB | Recommended default. Best balance of size, speed and quality. |
UD-Q4_K_XL
|
2.9 GB | Dynamic. Embeddings and output kept at Q8_0 for higher quality at a Q4 footprint. |
Q5_K_S
|
3.0 GB | Higher quality. |
Q5_K_M
|
3.1 GB | Higher quality, low loss. |
Q6_K
|
3.5 GB | Near lossless. |
Q8_0
|
4.5 GB | Effectively lossless, reference quality. |
Pick the largest file that fits your (V)RAM with room for context.
Q4_K_MorUD-Q4_K_XLis the sweet spot for most setups;Q6_KorQ8_0for maximum fidelity.
Get started
Run Qwen3.5 4B locally with:
-
Atomic Chat
:
the easiest path. Open the app, search
AtomicChat/qwen35-4b-GGUF, pick a quant, hit Use this model . -
llama.cpp:
llama-server -hf AtomicChat/qwen35-4b-GGUF:Q4_K_M --jinja -c 8192 -
Ollama:
ollama run hf.co/AtomicChat/qwen35-4b-GGUF:Q4_K_M - LM Studio / Jan: search the repo id, download any quant.
Best practices
| Parameter | Value |
|---|---|
| temperature | 0.7 |
| top_p | 0.8 |
| top_k | 20 |
| min_p | 0.0 |
| presence_penalty | 1.5 |
| repetition_penalty | 1.0 |
Qwen's recommended Instruct (non-thinking) settings. Thinking mode for general tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0.
Run in llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
./llama.cpp/build/bin/llama-server \
-hf AtomicChat/qwen35-4b-GGUF:UD-Q4_K_XL \
--jinja -ngl 99 -c 8192 -fa on
How these were made
-
Download
Qwen/Qwen3.5-4B(original weights). - Convert to f16 GGUF with llama.cpp .
-
Build an importance matrix over
calibration_datav3(100 chunks). -
Quantize the full ladder with
--imatrix. -
UD-Q4_K_XLadditionally pins the token-embedding and output tensors toQ8_0.
License
Released by Qwen under the Apache 2.0 license. Quantized by Atomic Chat.
Runs of AtomicChat Qwen3.5-4B-GGUF on huggingface.co
8.2K
Total runs
455
24-hour runs
1.5K
3-day runs
2.9K
7-day runs
3.4K
30-day runs
More Information About Qwen3.5-4B-GGUF huggingface.co Model
Url of Qwen3.5-4B-GGUF
Qwen3.5-4B-GGUF huggingface.co Url
Provider of Qwen3.5-4B-GGUF huggingface.co
Other API from AtomicChat
Updated:October 14 2026
Total runs:
296.3K
Run Growth:
251.0K
Growth Rate:
97.17%
Total runs:
234.6K
Run Growth:
-98.8K
Growth Rate:
-42.09%
Total runs:
119.4K
Run Growth:
102.8K
Growth Rate:
94.36%
Total runs:
53.5K
Run Growth:
-16.9K
Growth Rate:
-30.35%
Total runs:
53.0K
Run Growth:
-17.2K
Growth Rate:
-31.22%
Total runs:
52.9K
Run Growth:
-17.1K
Growth Rate:
-31.06%
Total runs:
48.2K
Run Growth:
-213.8K
Growth Rate:
-459.90%
Total runs:
45.9K
Run Growth:
-15.6K
Growth Rate:
-34.40%
Total runs:
25.0K
Run Growth:
-66.9K
Growth Rate:
-264.09%
Total runs:
14.0K
Run Growth:
7.4K
Growth Rate:
54.69%
Total runs:
13.8K
Run Growth:
9.4K
Growth Rate:
72.11%
Total runs:
12.5K
Run Growth:
2.8K
Growth Rate:
100.00%
Total runs:
11.1K
Run Growth:
5.9K
Growth Rate:
53.88%
Total runs:
10.0K
Run Growth:
5.7K
Growth Rate:
74.53%
Total runs:
8.1K
Run Growth:
2.0K
Growth Rate:
84.29%
Total runs:
7.9K
Run Growth:
5.5K
Growth Rate:
72.76%
Total runs:
7.5K
Run Growth:
4.7K
Growth Rate:
63.29%
Total runs:
7.2K
Run Growth:
2.7K
Growth Rate:
37.78%
Total runs:
6.3K
Run Growth:
3.6K
Growth Rate:
57.21%
Total runs:
6.1K
Run Growth:
759
Growth Rate:
25.30%
Total runs:
5.6K
Run Growth:
1.2K
Growth Rate:
21.24%
Total runs:
4.7K
Run Growth:
2.7K
Growth Rate:
59.51%
Total runs:
3.6K
Run Growth:
1.4K
Growth Rate:
42.84%
Total runs:
3.5K
Run Growth:
1.5K
Growth Rate:
42.52%
Total runs:
3.4K
Run Growth:
2.1K
Growth Rate:
77.34%
Total runs:
3.2K
Run Growth:
423
Growth Rate:
54.23%
Total runs:
2.5K
Run Growth:
124
Growth Rate:
4.89%
Total runs:
2.1K
Run Growth:
724
Growth Rate:
32.67%
Total runs:
2.0K
Run Growth:
-910
Growth Rate:
-46.52%
Total runs:
1.9K
Run Growth:
1.8K
Growth Rate:
100.00%
Total runs:
1.7K
Run Growth:
657
Growth Rate:
37.65%
Total runs:
1.7K
Run Growth:
142
Growth Rate:
8.33%
Total runs:
1.6K
Run Growth:
105
Growth Rate:
7.61%
Total runs:
1.6K
Run Growth:
1.6K
Growth Rate:
100.00%
Total runs:
1.6K
Run Growth:
514
Growth Rate:
98.66%
Total runs:
1.5K
Run Growth:
-86
Growth Rate:
-5.68%
Total runs:
1.4K
Run Growth:
-3.0K
Growth Rate:
-204.14%
Total runs:
1.4K
Run Growth:
348
Growth Rate:
25.18%
Total runs:
1.3K
Run Growth:
1.1K
Growth Rate:
88.93%
Total runs:
1.2K
Run Growth:
1.2K
Growth Rate:
100.00%
Total runs:
1.1K
Run Growth:
54
Growth Rate:
5.03%
Total runs:
1.0K
Run Growth:
1.0K
Growth Rate:
100.00%
Total runs:
1.0K
Run Growth:
1.0K
Growth Rate:
100.00%
Total runs:
998
Run Growth:
458
Growth Rate:
50.27%
Total runs:
910
Run Growth:
347
Growth Rate:
37.72%
Total runs:
872
Run Growth:
100
Growth Rate:
11.59%
Total runs:
812
Run Growth:
223
Growth Rate:
27.26%
Total runs:
735
Run Growth:
-581
Growth Rate:
-78.09%
Total runs:
732
Run Growth:
-6.1K
Growth Rate:
-829.55%
Total runs:
703
Run Growth:
169
Growth Rate:
24.28%
Total runs:
644
Run Growth:
-131
Growth Rate:
-21.13%
Total runs:
618
Run Growth:
241
Growth Rate:
36.08%
Total runs:
602
Run Growth:
602
Growth Rate:
100.00%
Total runs:
572
Run Growth:
142
Growth Rate:
25.22%
Total runs:
524
Run Growth:
0
Growth Rate:
0.00%
Total runs:
480
Run Growth:
-172
Growth Rate:
-35.76%
Total runs:
455
Run Growth:
437
Growth Rate:
100.00%
Total runs:
442
Run Growth:
-35
Growth Rate:
-7.95%
Total runs:
434
Run Growth:
30
Growth Rate:
7.03%
Total runs:
429
Run Growth:
-426
Growth Rate:
-99.30%
Total runs:
423
Run Growth:
22
Growth Rate:
4.74%
Total runs:
357
Run Growth:
0
Growth Rate:
0.00%
Total runs:
349
Run Growth:
-642
Growth Rate:
-181.36%
Total runs:
325
Run Growth:
15
Growth Rate:
4.70%
Total runs:
295
Run Growth:
-130
Growth Rate:
-44.67%
Total runs:
272
Run Growth:
-734
Growth Rate:
-269.85%
Total runs:
233
Run Growth:
-36
Growth Rate:
-15.79%
Total runs:
231
Run Growth:
-32
Growth Rate:
-14.29%
Total runs:
226
Run Growth:
66
Growth Rate:
28.70%
Total runs:
222
Run Growth:
-298
Growth Rate:
-129.57%
Total runs:
214
Run Growth:
-351
Growth Rate:
-161.75%
Total runs:
209
Run Growth:
36
Growth Rate:
15.65%
Total runs:
193
Run Growth:
-61
Growth Rate:
-30.81%
Total runs:
186
Run Growth:
-211
Growth Rate:
-115.30%
Total runs:
171
Run Growth:
-95
Growth Rate:
-56.21%
Total runs:
167
Run Growth:
3
Growth Rate:
1.82%
Total runs:
165
Run Growth:
-55
Growth Rate:
-32.35%
Total runs:
162
Run Growth:
-21
Growth Rate:
-13.29%
Total runs:
155
Run Growth:
-133
Growth Rate:
-87.50%
Total runs:
150
Run Growth:
-244
Growth Rate:
-153.46%
Total runs:
143
Run Growth:
-13
Growth Rate:
-10.74%
Total runs:
128
Run Growth:
-88
Growth Rate:
-67.18%
Total runs:
123
Run Growth:
-192
Growth Rate:
-169.91%
Total runs:
120
Run Growth:
-26
Growth Rate:
-23.01%
Total runs:
117
Run Growth:
117
Growth Rate:
100.00%
Total runs:
117
Run Growth:
104
Growth Rate:
88.89%
Total runs:
96
Run Growth:
-180
Growth Rate:
-183.67%
Total runs:
94
Run Growth:
-192
Growth Rate:
-202.11%
Total runs:
94
Run Growth:
82
Growth Rate:
87.23%
Total runs:
91
Run Growth:
-17
Growth Rate:
-19.32%
Total runs:
90
Run Growth:
0
Growth Rate:
0.00%
Total runs:
90
Run Growth:
-35
Growth Rate:
-37.23%
Total runs:
88
Run Growth:
-115
Growth Rate:
-121.05%
Total runs:
80
Run Growth:
-1
Growth Rate:
-1.49%
Total runs:
79
Run Growth:
-130
Growth Rate:
-164.56%
Total runs:
75
Run Growth:
-67
Growth Rate:
-94.37%