abacusai / Smaug-Mini

1. Model Introduction

Smaug-Mini is an agentic finetune of Qwen3.8-27B , finetuned by Abacus.AI. It improves end-to-end agentic tool use and automation performance (+4.5 on AutomationBench, +17.1 on JobBench, +2.0 overall on LiveBench, +2.5 on IFBench) while holding the base model's general capabilities at parity (GPQA-diamond at the official number, LiveBench coding and agentic-coding at board parity). The weights load exactly like the official release: same layout, same Qwen3_5ForConditionalGeneration architecture with the vision tower intact, same 262,144-token context, same reasoning interface with xhigh / medium / low effort levels.

This card describes the training approach and the evaluation results. Dataset contents are not disclosed; training data consists of multi-turn, tool-using automation episodes generated and verified against executable environments.

2. Model Summary

3. Evaluation

Smaug-Mini vs its base

Higher is better. Base points from the Qwen3.8-27B model card and livebench.ai.

LiveBench category profile

Scores 0–100; overall = mean of the seven category averages. Both models' scores are the published livebench.ai leaderboard entries (Smaug-Mini listed under the finetunes filter).

4. Training Approach

Smaug-Mini was trained to make multi-turn tool use and automation episodes more reliable end to end. It is trained with on-policy reinforcement learning (GRPO) over multi-turn, tool-using automation episodes with verified, outcome-based rewards - an episode is rewarded for measurably completing its task in the environment. Training is a LoRA adapter over the language trunk only, merged into the base weights as a full delta; the vision tower and multimodal projection are bitwise-identical to the base release.

One deployment note: the multi-token-prediction (MTP) head is inherited from the base model and was not retrained against the updated trunk. Speculative decoding via MTP should be left off; standard decoding is unaffected.

5. Known Behaviors and Limitations

Smaug-Mini redistributes deliberation rather than adding it: more reasoning per action but fewer actions per episode, finishing passing episodes about three steps sooner at essentially unchanged total reasoning volume. The clearest shift is in failure behavior — episodes that burn the entire step budget without finishing drop from 3.4% to 1.0%. Refusals are zero for both models, and the general-capability benchmarks in §3 show the deliberation shift does not come at the cost of base skills.

6. Deployment

Smaug-Mini serves as a drop-in replacement for Qwen3.8-27B on any stack that supports the base model. Reference vLLM invocation:

vllm serve abacusai/Smaug-Mini \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

Recommended sampling for agentic use: temperature 1.0, top_p 0.95, reasoning effort xhigh (the default). Recent vLLM versions return the chain-of-thought in the reasoning field of the message; when replaying conversation history, pass reasoning back under both reasoning and reasoning_content keys for portability across serving versions. Leave MTP-based speculative decoding off (§4).

7. License

Apache 2.0, inherited from Qwen3.8-27B.

8. Citation

@misc{abacusai2026smaugmini,
  title  = {Smaug-Mini},
  author = {Abacus.AI},
  year   = {2026},
  note   = {Agentic fine-tune of Qwen/Qwen3.8-27B},
  url    = {https://huggingface.co/abacusai/Smaug-Mini}
}

The Smaug line and the DPO-Positive method behind it:

@article{pal2024smaug,
  title={Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive},
  author={Pal, Arka and Karkhanis, Deep and Dooley, Samuel and
          Roberts, Manley and Naidu, Siddartha and White, Colin},
  journal={arXiv preprint arXiv:2402.13228},
  year={2024}
}

9. Contact Us

If you have any questions, please reach out at Abacus.AI .