Total runs:
1.5K
Run Growth:
-485
Growth Rate:
-31.72%
Sponsored by
Agentkit
DJLougen / Ornstein-26B-A4B-it-FP8
Model's Last Updated: April 11 2026
image-text-to-text
Introduction of Ornstein-26B-A4B-it-FP8
Model Details of Ornstein-26B-A4B-it-FP8
Ornstein-26B-A4B-it-FP8
FP8 (8-bit floating point) quantization of DJLougen/Ornstein-26B-A4B-it - a reasoning-focused fine-tune of Gemma 4 26B-A4B-it trained on 1,229 high-quality reasoning traces curated through a custom Drift Diffusion Modeling (DDM) pipeline.
Support This Work
I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded - balancing GPU costs against a student budget. If my models have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.
About This Quantization
This is the FP8 quantization of Ornstein-26B-A4B-it via bitsandbytes. FP8 cuts memory usage roughly in half compared to FP16/BF16 while preserving near-lossless quality - the best option when you want maximum fidelity at reduced VRAM.
| Format | Notes |
|---|---|
| FP8 | 8-bit floating point via bitsandbytes. Near-lossless quality. Requires GPU with FP8 support (Ada Lovelace, Hopper, Blackwell). |
Other formats:
- Full precision: DJLougen/Ornstein-26B-A4B-it
- NVIDIA FP4: DJLougen/Ornstein-26B-A4B-it-NVFP4
- GGUF: DJLougen/Ornstein-26B-A4B-it-GGUF
- MLX 4-bit: DJLougen/Ornstein-26B-A4B-it-MLX-4bit
- MLX 8-bit: DJLougen/Ornstein-26B-A4B-it-MLX-8bit
What Makes Ornstein Different
Unlike typical reasoning fine-tunes that use large volumes of synthetic data, Ornstein implements quality-over-quantity:
- Detects degenerate reasoning : Identifies "fake" reasoning that mimics thought without substance (hedging, restating, circling)
- Premium vs. Degenerate split : 799 premium traces + 430 selected degenerate traces = 1,229 total
- DDM AUC of 0.9705 separating premium from degenerate reasoning with 99.49% sensitivity
- Per-segment loss weighting : An Ornstein-Uhlenbeck stochastic process models quality evolution across each trace, generating loss weights that tell the trainer which parts matter most
The model uses extended thinking with multi-phase reasoning, self-correction, and verification before providing final answers.
Training Data at a Glance
- Drift Score Distribution: DDM drift scores for premium (blue) vs degenerate (red) traces, with the fitted threshold at 1.463 cleanly separating the two pools.
- Category Mix: Math-heavy (1,016), with code (124), science (45), and logic (44).
- Reasoning Depth: Premium traces average ~1,263 words of thinking vs ~281 for degenerate traces, which tend to be shallow repetition.
- Difficulty x Pool: The degenerate pool skews toward hard problems where models are most likely to loop or stall.
Training Configuration
| Parameter | Value |
|---|---|
| Base model |
unsloth/gemma-4-26B-A4B-it
|
| Architecture | Gemma 4 (MoE, 26B total, A4B active) |
| Method | LoRA (rank 32, alpha 32) |
| Dropout | 0.05 |
| Epochs | 1 |
| Training examples | 1,229 (DDM-curated) |
| Max sequence length | 8192 |
| Framework | Unsloth |
Data Quality Metrics
| Metric | Value |
|---|---|
| Total examples | 1,229 |
| Mean thinking depth | ~1,667 words |
| Self-correction present | 100% of traces |
| Verification present | 100% of traces |
| Exploration present | 100% of traces |
| Quality gate pass rate | 100% |
Recommended Settings
| Parameter | Suggested Value |
|---|---|
| Temperature | 0.6 |
| Top-P | 0.95 |
| Max tokens | 8192 |
| Repeat penalty | 1.1 |
Intended Use
Designed for tasks requiring structured, multi-step reasoning:
- Mathematics
- Logic problems
- Code analysis
- Scientific problems
- Complex question answering
Limitations
- Single epoch training on 1,229 examples means the model retains most base Gemma 4 behavior; the fine-tune primarily shapes reasoning style rather than injecting new knowledge
- Language scope: DDM pipeline optimized for English; other languages reflect base model performance
- Extended thinking can occasionally loop on adversarial or highly ambiguous prompts
Citation
@misc{ornstein26b,
author = {DJLougen},
title = {Ornstein-26B-A4B-it: DDM-Curated Reasoning Fine-Tune of Gemma 4 26B-A4B},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/DJLougen/Ornstein-26B-A4B-it}
}
Links
- Full-precision model : DJLougen/Ornstein-26B-A4B-it
- GGUF quantizations : DJLougen/Ornstein-26B-A4B-it-GGUF
- NVIDIA FP4 : DJLougen/Ornstein-26B-A4B-it-NVFP4
- Base model : Google Gemma 4 26B-A4B-it | unsloth/gemma-4-26B-A4B-it
License
Apache 2.0
Runs of DJLougen Ornstein-26B-A4B-it-FP8 on huggingface.co
2
Total runs
0
24-hour runs
-2
3-day runs
-31
7-day runs
-84
30-day runs
More Information About Ornstein-26B-A4B-it-FP8 huggingface.co Model
Url of Ornstein-26B-A4B-it-FP8
Ornstein-26B-A4B-it-FP8 huggingface.co Url
Provider of Ornstein-26B-A4B-it-FP8 huggingface.co
Other API from DJLougen
Updated:April 04 2026
Total runs:
1.4K
Run Growth:
-3.4K
Growth Rate:
-234.70%
Total runs:
808
Run Growth:
332
Growth Rate:
41.09%
Total runs:
687
Run Growth:
-301
Growth Rate:
-43.81%
Total runs:
518
Run Growth:
-269
Growth Rate:
-54.23%
Total runs:
512
Run Growth:
-2.1K
Growth Rate:
-404.69%
Total runs:
395
Run Growth:
-3.0K
Growth Rate:
-749.50%
Total runs:
242
Run Growth:
-6.4K
Growth Rate:
-2660.33%
Total runs:
228
Run Growth:
-634
Growth Rate:
-278.07%
Total runs:
205
Run Growth:
27
Growth Rate:
13.17%
Total runs:
186
Run Growth:
-3.0K
Growth Rate:
-1627.42%
Total runs:
185
Run Growth:
-233
Growth Rate:
-125.95%
Total runs:
179
Run Growth:
-1.7K
Growth Rate:
-975.98%
Total runs:
154
Run Growth:
-2.1K
Growth Rate:
-1349.35%
Total runs:
150
Run Growth:
75
Growth Rate:
50.00%
Total runs:
141
Run Growth:
-1.7K
Growth Rate:
-1226.95%
Total runs:
137
Run Growth:
-306
Growth Rate:
-223.36%
Total runs:
129
Run Growth:
-3.2K
Growth Rate:
-2482.17%
Total runs:
122
Run Growth:
-984
Growth Rate:
-806.56%
Total runs:
119
Run Growth:
-1.3K
Growth Rate:
-1058.82%
Total runs:
117
Run Growth:
-195
Growth Rate:
-166.67%
Total runs:
111
Run Growth:
-254
Growth Rate:
-228.83%
Total runs:
111
Run Growth:
-797
Growth Rate:
-718.02%
Total runs:
106
Run Growth:
-930
Growth Rate:
-877.36%
Total runs:
96
Run Growth:
-303
Growth Rate:
-315.63%
Total runs:
92
Run Growth:
-1.3K
Growth Rate:
-1421.74%
Total runs:
91
Run Growth:
24
Growth Rate:
26.37%
Total runs:
89
Run Growth:
89
Growth Rate:
100.00%
Total runs:
85
Run Growth:
45
Growth Rate:
52.94%
Total runs:
73
Run Growth:
-142
Growth Rate:
-189.33%
Total runs:
72
Run Growth:
-192
Growth Rate:
-266.67%
Total runs:
64
Run Growth:
9
Growth Rate:
14.06%
Total runs:
62
Run Growth:
-174
Growth Rate:
-263.64%
Total runs:
59
Run Growth:
-960
Growth Rate:
-1627.12%
Total runs:
58
Run Growth:
-21
Growth Rate:
-36.84%
Total runs:
56
Run Growth:
-368
Growth Rate:
-657.14%
Total runs:
55
Run Growth:
-14
Growth Rate:
-25.45%
Total runs:
54
Run Growth:
54
Growth Rate:
100.00%
Total runs:
53
Run Growth:
-811
Growth Rate:
-1530.19%
Total runs:
52
Run Growth:
0
Growth Rate:
0.00%
Total runs:
49
Run Growth:
49
Growth Rate:
100.00%
Total runs:
48
Run Growth:
38
Growth Rate:
79.17%
Total runs:
47
Run Growth:
-492
Growth Rate:
-1046.81%
Total runs:
43
Run Growth:
-445
Growth Rate:
-1034.88%
Total runs:
37
Run Growth:
-389
Growth Rate:
-1051.35%
Total runs:
32
Run Growth:
32
Growth Rate:
100.00%
Total runs:
30
Run Growth:
-201
Growth Rate:
-670.00%
Total runs:
28
Run Growth:
-28
Growth Rate:
-100.00%
Total runs:
28
Run Growth:
-45
Growth Rate:
-160.71%
Total runs:
25
Run Growth:
-7
Growth Rate:
-28.00%
Total runs:
23
Run Growth:
-81
Growth Rate:
-352.17%
Total runs:
22
Run Growth:
-6
Growth Rate:
-27.27%
Total runs:
21
Run Growth:
-201
Growth Rate:
-957.14%
Total runs:
21
Run Growth:
-3
Growth Rate:
-14.29%
Total runs:
20
Run Growth:
-21
Growth Rate:
-105.00%
Total runs:
18
Run Growth:
-6
Growth Rate:
-33.33%
Total runs:
18
Run Growth:
-517
Growth Rate:
-2872.22%
Total runs:
18
Run Growth:
-135
Growth Rate:
-750.00%
Total runs:
12
Run Growth:
4
Growth Rate:
33.33%
Total runs:
12
Run Growth:
-10
Growth Rate:
-83.33%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8
Run Growth:
-28
Growth Rate:
-350.00%
Total runs:
5
Run Growth:
-138
Growth Rate:
-2760.00%
Total runs:
3
Run Growth:
-1
Growth Rate:
-33.33%
Total runs:
1
Run Growth:
-105
Growth Rate:
-10500.00%
Total runs:
1
Run Growth:
-3
Growth Rate:
-300.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%