PursuitOfDataScience / Argonne-Qwen1.5-0.5B-think

Argonne-Qwen1.5-0.5B-think

A 0.46B-parameter chain-of-thought reasoner : the base model Qwen1.5-0.5B fine-tuned with a three-stage reasoning recipe (general SFT → DPO → CoT-SFT). It produces an explicit <think> … </think> reasoning trace followed by a final \boxed{} answer on math word problems.

The point of this model is to show how much math-reasoning ability a well-pretrained sub-1B base can be given by a light, fully open post-training recipe. It is a compact research reasoner — useful for studying test-time compute on small models — not a production-grade or competition-math solver.

Benchmarks (honest, held-out)

Evaluated on contamination-free elementary math-word-problem benchmarks — SVAMP, ASDiv, MAWPS (clean) and GSM-Plus (semi-clean, adversarial) — that appear in none of the training stages. n=500/set , with-think decoding. Metrics: greedy / +budget-forcing / self-consistency@32 / pass@32 (%).

Read plainly: single-shot greedy is ~24%, but the model can solve far more than it reliably answers in one pass — pass@32 ≈ 80% on the clean sets. Cheap test-time compute closes much of that gap (self-cons ≈ 48%). These are absolute-modest numbers for a 0.46B model; it is a small reasoner, and the pass@K headroom is the interesting part.

Note on GSM8K: the CoT training mix overlaps GSM8K, so GSM8K is not a valid held-out benchmark for this model. Use the clean SVAMP / ASDiv / MAWPS numbers above as the honest signal.

How it was trained

Base Qwen1.5-0.5B → three stages (base-agnostic harness reasoning/reason_control/ in the training repo ):

1. General SFT on HuggingFaceH4/ultrachat_200k (45k examples, LR 2e-5).
2. DPO preference alignment on argilla/dpo-mix-7k (LR 5e-6, β 0.1).
3. CoT-SFT on a <think>…</think> + \boxed{} math-reasoning mix (LR 1e-5, 1 epoch).

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

mid = "PursuitOfDataScience/Argonne-Qwen1.5-0.5B-think"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16).to("cuda").eval()

msgs = [{"role": "user", "content": "Ana has 3 boxes with 12 pencils each. She gives away 8. How many are left?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# -> <think> ... 3 * 12 = 36 ... 36 - 8 = 28 ... </think>  Answer: 28

The model was CoT-trained to open a <think> block itself; if it doesn't, prefill "<think>\n" after the generation prompt to force a reasoning trace. Uses the standard ChatML format ( <|im_start|> / <|im_end|> ).

Getting the most out of it (test-time compute)

Single-shot greedy leaves a lot on the table (pass@32 ≈ 80% vs greedy ≈ 24%). The dominant single-shot failure is non-termination (the <think> block runs long), so the cheap wins are:

• Self-consistency: sample K≈16–32 traces ( temperature 0.8, top_p 0.95 ), majority-vote the \boxed{} answers → ≈ 2× greedy at zero training cost (the 47.9% column above).
• Budget-forcing: force-close </think> past a token budget so the model commits to an answer.

Limitations

• 0.46B parameters — a small reasoner. Absolute accuracy on clean grade-school math is ~24% greedy / ~48% self-consistency; expect errors, especially on multi-step or adversarial problems.
• Non-termination: greedy traces sometimes fail to close <think> — budget-forcing / self-consistency mitigate this.
• Scope: trained and evaluated on English elementary math word problems + general chat. Not a general-purpose or competition-math model.
• GSM8K is not a valid benchmark here (train-mix overlap) — use the clean numbers above.

Base model & license

Fine-tuned from Qwen/Qwen1.5-0.5B (Apache-2.0) and released under Apache-2.0; please also observe the base model's terms.

Provenance

Training + evaluation code: PursuitOfDataScience/ArgonneAI ( reasoning/reason_control/ recipe; reasoning/clean_eval.py honest evaluator).

Citation

@misc{argonne_qwen05b_think_2026,
  title  = {Argonne-Qwen1.5-0.5B-think: a compact chain-of-thought reasoner},
  author = {PursuitOfDataScience},
  year   = {2026},
  url    = {https://huggingface.co/PursuitOfDataScience/Argonne-Qwen1.5-0.5B-think}
}