birgermoell / Qwen3.5-4B-EU

Qwen3.5-4B-EU

A European post-training of Qwen/Qwen3.5-4B , specialised for the OpenEuroLLM target languages. The strongest model in our compact EU lineup (2B + 4B). Defaults to fast no-think answers and keeps a /think reasoning toggle.

Built with a reproducible SFT → SimPO pipeline on European multilingual data.

Evaluation

OpenEuroLLM EU eval holdouts (38 European languages × 10 task buckets, deterministic scoring, full dev split = 1368 rows):

The 4B base is already strong, so the headroom is smaller than for the 2B (where the same pipeline gave +12.9). Gains concentrate on several EU languages — e.g. sr 47→72, fi 50→72, el 42→61, mt 44→61, sl 50→61, ru 50→61, fr 50→61, sv 50→56. (Per-language cells are small-sample/noisy; some languages move down. The overall +2.7 is the robust signal.)

For the smaller sibling see Qwen3.5-2B-EU (32.7→45.6, +12.9).

Pipeline

1. SFT on a European instruction mix: ~85% general EU instructions (no-think, direct answers)
   • ~15% reasoning traces (think-format) — defaults to fast answers, keeps a sharp /think path. Full fine-tune, bf16.
2. SimPO (reference-free preference optimisation) on European exam-MCQ preference pairs (~35 languages). Final preference accuracy ≈ 0.88.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer  # transformers >= 5.5 (qwen3_5 arch)
import torch

mid = "birgermoell/Qwen3.5-4B-EU"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16, device_map="auto")

msgs = [{"role": "user", "content": "Vad är meningen med livet? Svara kort."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=200)[0, ids.shape[1]:], skip_special_tokens=True))

• No-think (default): fast, direct answers — best for interactive use.
• Thinking: add /think to the user turn (or enable_thinking=True ) for step-by-step reasoning.

On-device status

• bf16 (this repo): runs via transformers / vLLM. Recommended today.
• GGUF (llama.cpp / Ollama): ⚠️ not supported yet — llama.cpp drops Qwen3.5's hybrid linear-attention (Gated Delta Net) layers during conversion (load fails). Blocked upstream.
• MLX (Apple): ⚠️ not supported yet — mlx-lm raises Model type qwen3_5 not supported .

Qwen3.5's hybrid architecture is very new (Mar 2026); the on-device quantization runtimes haven't added support yet. Phone deployment works once llama.cpp / MLX add Gated-Delta-Net support.

License

Apache-2.0 (inherits the Qwen3.5-4B base license). Built within OpenEuroLLM.