artindnr / BlueBerry-1

🫐 Blueberry-1

Blueberry-1 is a fine-tuned version of openai/gpt-oss-120b , trained to produce high-quality Farsi (Persian) reasoning traces and to perform multilingual chain-of-thought reasoning .

Blueberry-1 is the larger, more capable successor to Strawberry-1 , our ~20B model that set out to prove open-source LLMs could reason natively in Farsi rather than defaulting to English chain-of-thought. Strawberry validated the approach at a smaller scale; Blueberry takes the same recipe and applies it to a ~6x larger base model with additional training, yielding substantially stronger reasoning quality in both Farsi and English.

To the best of our knowledge, Blueberry-1 is the strongest open-source LLM capable of generating high-quality Farsi reasoning chains , extending the Strawberry line's goal of native multilingual reasoning to a much more capable model class.

Model Details

• Base model: openai/gpt-oss-120b (117B parameters)
• Architecture: gpt_oss
• Fine-tuned by: artindnr
• License: Apache 2.0
• Languages: Farsi (Persian), English, and multilingual reasoning support
• Model type: Causal decoder-only language model with reasoning ("thinking") traces
• Predecessor: Strawberry-1 (built on gpt-oss-20b )

What's New

Most open reasoning models today generate their chain-of-thought almost exclusively in English, even when the final answer is requested in another language. This was the founding motivation behind the Strawberry series, and Blueberry-1 carries it forward at a much larger scale:

• Generate coherent, high-quality reasoning traces in Farsi , not just Farsi answers
• Reason natively across multiple languages rather than silently falling back to English
• Preserve and extend the general instruction-following and reasoning ability of the gpt-oss-120b base model
• Deliver a meaningful quality jump over Strawberry-1 thanks to the larger base model and additional training

Training

Blueberry-1 was trained using a mix of fine-tuning strategies — including full fine-tuning and LoRA experiments — on top of gpt-oss-120b , building on lessons learned from the Strawberry series. The version released here is the fully fine-tuned (merged, dense-weights) checkpoint , not a LoRA adapter.

How to Use

Blueberry-1 uses the gpt-oss chat template (Harmony format) shipped with the base model, so it works with 🤗 Transformers.

Installation

pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install trl peft transformers kernels

Generation

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "artindnr/blueberry-1"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

REASONING_LANGUAGE = "English"  # e.g. "English", "Farsi", "Persian"
SYSTEM_PROMPT = f"reasoning language: {REASONING_LANGUAGE}"
USER_PROMPT = "تو کی هستی و اسمت چیه؟"

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.6,
    do_sample=True,
)

print(tokenizer.decode(outputs[0]))

This prints the full Harmony-formatted output, including the analysis (reasoning) and final (answer) channels and their special tokens. To get just the plain-text final answer, decode with skip_special_tokens=True and parse out the final channel, or use tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True) to only decode the newly generated tokens.

Reasoning in a specific language

Set reasoning language: <Language> as the system message content to control the language of the reasoning trace (the analysis channel), independent of the language the user writes in. For example, setting REASONING_LANGUAGE = "Farsi" will produce a Farsi reasoning trace even for a prompt in another language.

Note that the model's default chat template also auto-populates a Harmony-format preamble (identity, knowledge cutoff, current date, reasoning effort, valid channels) ahead of your system/developer message — you don't need to set these yourself.

Relationship to Strawberry-1

Blueberry-1 is not a from-scratch project — it's the next step in the same line of work as Strawberry-1:

If you need a lighter-weight model or are resource-constrained, Strawberry-1 remains a solid option built on the same principles. Blueberry-1 is for use cases that need stronger reasoning quality and can accommodate a larger model.

Intended Use

Blueberry-1 is intended for:

• Research and experimentation on multilingual and Farsi-language reasoning
• Building Farsi-language assistants, tutoring tools, and reasoning-heavy applications
• General-purpose