Total runs:
626
Run Growth:
156
Growth Rate:
24.92%
Sponsored by
Vmake Labs Video Enhancer
h3rb3rn / smollm3-expert-coder-3b
Model's Last Updated: September 12 2026
text-generation
Introduction of smollm3-expert-coder-3b
Model Details of smollm3-expert-coder-3b
MoE Sovereign Systems-Programming & Code-Synthesis Expert 3B -- SmolLM3 (
smollm3-expert-coder-3b
)
Model Summary
smollm3-expert-coder-3b
is a LoRA fine-tune of the text-decoder of
Qwen3.5-4B
, specialized as the coder domain expert within the MoE Sovereign compound-AI system.
You are a high-assurance systems-programming and code-synthesis expert (moe-expert-coder-4b) specialized in Rust, C++, Python, and Go. Produce precise, compiler-checked code and minimal atomic diffs. Uphold memory-safety invariants strictly -- correct ownership, correct lock-free memory ordering (acquire/release pairing), no data races. Flag any construct you cannot verify as sound rather than guessing.
Base Architecture
SmolLM3-3B is a genuinely open-source dense Transformer (weights, training data, and training code all publicly documented by HuggingFaceTB) -- distinguishing this Spur-2 track from the open-weight-only Qwen3.5 base used in the parallel 4B expert line.
Training Configuration
| Parameter | Value |
|---|---|
| Method |
LoRA (rank 16, alpha 32, dropout 0.05), targeting
q/k/v/o_proj
+
gate/up/down_proj
|
| Trainable parameters | 30,228,480 of 3,105,327,104 (0.97%) |
| Epochs | 3 |
| Effective batch size | 128 (micro-batch 4 x 8 GPUs x grad-accum 4) |
| Learning rate | 1.5e-5 |
| Training sequence length | 4,096 tokens |
| Optimizer sharding | DeepSpeed ZeRO-2, bf16 |
| Compute | EuroHPC LUMI-G, 8x AMD Instinct MI250X GCDs, ROCm |
| Training examples | 2,294 curated instruction/response pairs |
Observed Training Trajectory
Training loss over the run (representative logged steps): 1.536 -> 1.477 -> 1.385. Smooth, monotonic decline consistent with genuine generalization, not memorization.
Prompt Format
ChatML:
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
{response}<|im_end|>
System Prompt
You are a high-assurance systems-programming and code-synthesis expert (moe-expert-coder-4b) specialized in Rust, C++, Python, and Go. Produce precise, compiler-checked code and minimal atomic diffs. Uphold memory-safety invariants strictly -- correct ownership, correct lock-free memory ordering (acquire/release pairing), no data races. Flag any construct you cannot verify as sound rather than guessing.
Available Formats
| File | Notes |
|---|---|
smollm3-expert-coder-3b-Q4_K_M.gguf
|
Recommended for single/multi-GPU deployment |
smollm3-expert-coder-3b-Q8_0.gguf
|
Higher-fidelity reference quantization |
Hardware Guidance
SmolLM3-3B's native context window is 65,536 tokens. On single 8GB GPUs cap
num_ctx
to 32,768 and use
f16
KV-cache on Maxwell-generation hardware (no Flash Attention support there).
Ollama
Modelfile
FROM ./smollm3-expert-coder-3b-Q4_K_M.gguf
SYSTEM """You are a high-assurance systems-programming and code-synthesis expert (moe-expert-coder-4b) specialized in Rust, C++, Python, and Go. Produce precise, compiler-checked code and minimal atomic diffs. Uphold memory-safety invariants strictly -- correct ownership, correct lock-free memory ordering (acquire/release pairing), no data races. Flag any construct you cannot verify as sound rather than guessing."""
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.2
PARAMETER num_ctx 32768
Limitations
- Does not execute code/queries/tools itself; outputs should be validated against the actual system before use.
- Specialized for its domain; general-purpose conversation is out of scope.
License
Apache 2.0, inherited from the SmolLM3-3B base model.
Runs of h3rb3rn smollm3-expert-coder-3b on huggingface.co
200
Total runs
0
24-hour runs
23
3-day runs
81
7-day runs
89
30-day runs
More Information About smollm3-expert-coder-3b huggingface.co Model
Url of smollm3-expert-coder-3b
smollm3-expert-coder-3b huggingface.co Url
Provider of smollm3-expert-coder-3b huggingface.co
Other API from h3rb3rn
Updated:September 12 2026
Total runs:
302
Run Growth:
63
Growth Rate:
20.86%
Total runs:
184
Run Growth:
60
Growth Rate:
32.61%
Total runs:
161
Run Growth:
49
Growth Rate:
30.43%
Total runs:
155
Run Growth:
155
Growth Rate:
100.00%
Total runs:
147
Run Growth:
10
Growth Rate:
6.80%
Total runs:
139
Run Growth:
64
Growth Rate:
46.04%
Total runs:
127
Run Growth:
58
Growth Rate:
45.67%
Total runs:
125
Run Growth:
20
Growth Rate:
16.00%
Total runs:
98
Run Growth:
0
Growth Rate:
0.00%
Total runs:
66
Run Growth:
37
Growth Rate:
56.06%
Total runs:
63
Run Growth:
33
Growth Rate:
52.38%
Total runs:
33
Run Growth:
19
Growth Rate:
57.58%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%