artindnr / ChatBerry-GGUF

🍰👑 ChatBerry GGUF

GGUF quantizations of artindnr/chatberry , the flagship release of the ChatBerry family — a direct-answer chat fine-tune of artindnr/strawberry-1 (itself built on openai/gpt-oss-20b ). ChatBerry's reasoning ("thinking") channel is disabled, so it responds directly instead of emitting a separate chain-of-thought trace.

These quants let you run ChatBerry locally with llama.cpp , Ollama, LM Studio, or any other GGUF-compatible runtime.

Files

If you're unsure which to pick: Q4_K_M is a solid default for most setups. Go up to Q5_K_M , Q6_K , or Q8_0 if you have the VRAM/RAM to spare and want maximum fidelity, and drop to Q3_K_M / Q2_K if you're tightly memory-constrained.

Usage

llama.cpp

./llama-cli -m chatberry.Q4_K_M.gguf -p "تو کی هستی و اسمت چیه؟" -n 512

Or serve it as an OpenAI-compatible endpoint:

./llama-server -m chatberry.Q4_K_M.gguf -c 4096

Ollama

Create a Modelfile :

FROM ./chatberry.Q4_K_M.gguf

Then:

ollama create chatberry -f Modelfile
ollama run chatberry

LM Studio

Download the .gguf file of your choice directly in LM Studio's model browser (search artindnr/chatberry-gguf ), or drop the file into your local models folder.

About ChatBerry

ChatBerry is the culmination of the iterative ChatBerry line (1.0 → 1.1 → 1.2 → ChatBerry), each release trained on more data than the last. It converts the reasoning behavior of strawberry-1 into a direct-answer chat model — no visible analysis/chain-of-thought channel, just a final response.

• Base model: artindnr/strawberry-1 (fine-tuned from openai/gpt-oss-20b , 21B parameters)
• Architecture: gpt_oss
• Languages: Farsi (Persian), English, and multilingual support
• Behavior: Reasoning disabled — responds directly without a separate analysis channel
• Format: GGUF, for use with llama.cpp and compatible runtimes

See the full model card for training details and intended use.

Chat Template

ChatBerry uses the gpt-oss chat template (Harmony format) inherited from its base model. Most GGUF runtimes ( llama.cpp , Ollama, LM Studio) apply this automatically when loading the model — no need to set a reasoning-language system message or parse separate channels; ChatBerry goes straight to its final answer.

Limitations

• Quantization introduces some precision loss versus the original bf16 weights — expect small quality differences across the tiers, most noticeable at Q2_K/Q3_K.
• ChatBerry trades away Strawberry-1's explicit chain-of-thought reasoning; for tasks that benefit from visible step-by-step reasoning, artindnr/strawberry-1 may be a better fit.
• Inherits the general capabilities and limitations of the gpt-oss-20b base model and strawberry-1 , including the possibility of hallucinated facts.
• No formal safety fine-tuning beyond what is inherited from the base model and Strawberry-1 has been applied; use appropriate safeguards in production settings.

License

Released under the Apache 2.0 license, consistent with the base gpt-oss-20b model, strawberry-1 , and chatberry .

Acknowledgements

• openai/gpt-oss-20b and artindnr/strawberry-1 for the base models
• artindnr/chatberry for the original fine-tune
• llama.cpp for the GGUF format and quantization tooling