JonathanMiddleton / Qwen3-Reranker-0.6B

huggingface.co
Total runs: 23
24-hour runs: 0
7-day runs: 1
30-day runs: -23
Model's Last Updated: August 19 2025
text-ranking

Introduction of Qwen3-Reranker-0.6B

Model Details of Qwen3-Reranker-0.6B

Qwen3-Reranker-0.6B-GGUF

🚨 REQUIRED Llama.cpp build: https://github.com/ngxson/llama.cpp/tree/xsn/qwen3_embd_rerank
This unmerged fix branch is mandatory to run Qwen3 reranking models. Other HF GGUF quantizations of the 0.6B reranker typically fail in mainline llama.cpp because they were not produced with this build. This quantization was produced with the above build and works.

Purpose

Multilingual text-reranking model in GGUF for efficient CPU/GPU inference with llama.cpp -compatible back-ends.
Parameters ≈ 0.6 B .

Note: Token embedding matrix and output tensors are left at FP16 across all quantizations.

Files
Filename Quant Size (bytes / MiB) Est. quality Δ vs FP16
Qwen3-Reranker-0.6B-F16.gguf FP16 1,197,634,048 B (1142.2 MiB) 0 (reference)
Qwen3-Reranker-0.6B-Q4_K_M.gguf Q4_K_M 396,476,032 B (378.1 MiB) TBD
Qwen3-Reranker-0.6B-Q5_K_M.gguf Q5_K_M 444,186,496 B (423.6 MiB) TBD
Qwen3-Reranker-0.6B-Q6_K.gguf Q6_K 494,878,880 B (472.0 MiB) TBD
Qwen3-Reranker-0.6B-Q8_0.gguf Q8_0 639,153,088 B (609.5 MiB) TBD
Upstream Source
  • Repo: Qwen/Qwen3-Reranker-0.6B
  • Commit: f16fc5d (2025-06-09)
  • License: Apache-2.0
Conversion & Quantization
# Convert safetensors → GGUF (FP16)
python convert_hf_to_gguf.py ~/models/local/Qwen3-Reranker-0.6B

# Quantize variants
EMB_OPT="--token-embedding-type F16 --leave-output-tensor"
for QT in Q4_K_M Q5_K_M Q6_K Q8_0; do
  llama-quantize $EMB_OPT Qwen3-Reranker-0.6B-F16.gguf Qwen3-Reranker-0.6B-${QT}.gguf $QT
done

Runs of JonathanMiddleton Qwen3-Reranker-0.6B on huggingface.co

23
Total runs
0
24-hour runs
1
3-day runs
1
7-day runs
-23
30-day runs

More Information About Qwen3-Reranker-0.6B huggingface.co Model

More Qwen3-Reranker-0.6B license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen3-Reranker-0.6B huggingface.co

Qwen3-Reranker-0.6B huggingface.co is an AI model on huggingface.co that provides Qwen3-Reranker-0.6B's model effect (), which can be used instantly with this JonathanMiddleton Qwen3-Reranker-0.6B model. huggingface.co supports a free trial of the Qwen3-Reranker-0.6B model, and also provides paid use of the Qwen3-Reranker-0.6B. Support call Qwen3-Reranker-0.6B model through api, including Node.js, Python, http.

JonathanMiddleton Qwen3-Reranker-0.6B online free

Qwen3-Reranker-0.6B huggingface.co is an online trial and call api platform, which integrates Qwen3-Reranker-0.6B's modeling effects, including api services, and provides a free online trial of Qwen3-Reranker-0.6B, you can try Qwen3-Reranker-0.6B online for free by clicking the link below.

JonathanMiddleton Qwen3-Reranker-0.6B online free url in huggingface.co:

https://huggingface.co/JonathanMiddleton/Qwen3-Reranker-0.6B

Qwen3-Reranker-0.6B install

Qwen3-Reranker-0.6B is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-Reranker-0.6B on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-Reranker-0.6B install, users can directly use Qwen3-Reranker-0.6B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen3-Reranker-0.6B install url in huggingface.co:

https://huggingface.co/JonathanMiddleton/Qwen3-Reranker-0.6B

Url of Qwen3-Reranker-0.6B

Provider of Qwen3-Reranker-0.6B huggingface.co

JonathanMiddleton
ORGANIZATIONS

Other API from JonathanMiddleton