auryn-macmillan / fuse-1-Lite-v2

Fuse-1 Lite v2

Fuse-1 Lite v2 is the trained successor of Akahisrr/fuse-1-Lite. It adds a v2 coding-expert pathway on top of the v1 model: bridge layers that map LFM2 representations to Qwen3.6-style coding experts, a router, an expert scale, and a coding gate — trained with KL distillation to the LFM2 teacher.

• Base architecture: Fuse3V2ForCausalLM (model_type = fuse3_v2)
• Trained: 550 steps (3 stages: foundation, representation mapping, router/refinement)
• Trainable params: ~253.7M (bridge, router, coding norm/gate, expert scale)
• Distillation: KL to LiquidAI/LFM2.5-2.6B (alpha=0.5)

⚠️ Custom code — trust_remote_code=True

This model uses a custom architecture and requires:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "auryn-macmillan/fuse-1-Lite-v2", trust_remote_code=True,
    torch_dtype="bfloat16",
)

trust_remote_code=True executes the custom Python code shipped in this repo (fuse3_model_v2.py). This is the same trust model as the upstream Akahisrr/fuse-1-Lite model. Please review the custom code before use and only run it in an environment where executing untrusted code is acceptable.

Quickstart

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "auryn-macmillan/fuse-1-Lite-v2"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, trust_remote_code=True
).cuda()
model.eval()
prompt = "Write a Python function to check if a number is prime."
inputs = tok(prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=128)
print(tok.decode(out[0]))

Files

• model.safetensors — merged full model weights (v1 base + trained v2 params)
• config.json — model_type=fuse3_v2, auto_map to fuse3_model_v2.py
• fuse3_model_v2.py, fuse3_model.py — custom model code (requires trust_remote_code)
• tokenizer files (tokenizer.json, tokenizer_config.json, chat_template.jinja)

Notes

• The coding experts are gated; set model.set_coding_enabled(False) to run the pure LFM2 pathway.
• Training was performed in an isolated container; this repository contains no training code or data.