Quantized GGUF versions of
Qwen/Qwen3-4B
— Alibaba's Qwen3 base model at the 4B parameter scale. Qwen3 represents a significant generational leap over Qwen2.5, with improved reasoning, coding, and instruction following packed into an efficient sub-5B footprint.
Available Files
File
Quant
Size
Use Case
Qwen3-4B-Q8_0.gguf
Q8_0
~4.5GB
Maximum quality
Qwen3-4B-Q6_K.gguf
Q6_K
~3.5GB
Near-lossless
Qwen3-4B-Q5_K_M.gguf
Q5_K_M
~3.1GB
High quality
Qwen3-4B-Q4_K_M.gguf
Q4_K_M
~2.6GB
Recommended default
Qwen3-4B-Q3_K_M.gguf
Q3_K_M
~2.1GB
Low VRAM
Qwen3-4B-IQ4_XS.gguf
IQ4_XS
~2.4GB
Imatrix 4-bit
Qwen3-4B-IQ3_XXS.gguf
IQ3_XXS
~1.8GB
Imatrix 3-bit
Qwen3-4B-IQ2_M.gguf
IQ2_M
~1.6GB
Imatrix 2-bit
Qwen3-4B-IQ1_S.gguf
IQ1_S
~1.2GB
Extreme compression
Qwen3-4B-fp16.gguf
FP16
~8.0GB
Full precision
imatrix.dat
—
—
Importance matrix
Usage
# llama.cpp
./llama-cli -m Qwen3-4B-Q4_K_M.gguf \
--ctx-size 8192 -n 512 \
-p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n"# Ollama
ollama run hf.co/DuoNeural/Qwen3-4B-GGUF:Q4_K_M
Qwen3-4B-GGUF huggingface.co is an AI model on huggingface.co that provides Qwen3-4B-GGUF's model effect (), which can be used instantly with this DuoNeural Qwen3-4B-GGUF model. huggingface.co supports a free trial of the Qwen3-4B-GGUF model, and also provides paid use of the Qwen3-4B-GGUF. Support call Qwen3-4B-GGUF model through api, including Node.js, Python, http.
Qwen3-4B-GGUF huggingface.co is an online trial and call api platform, which integrates Qwen3-4B-GGUF's modeling effects, including api services, and provides a free online trial of Qwen3-4B-GGUF, you can try Qwen3-4B-GGUF online for free by clicking the link below.
DuoNeural Qwen3-4B-GGUF online free url in huggingface.co:
Qwen3-4B-GGUF is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-4B-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-4B-GGUF install, users can directly use Qwen3-4B-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.