Total runs:
533.2K
Run Growth:
-258.0K
Growth Rate:
-48.38%
Sponsored by
Vmake Labs Video Enhancer
FINAL-Bench / Aether-7B-5Attn-it
Model's Last Updated: July 23 2026
text-generation
Introduction of Aether-7B-5Attn-it
Model Details of Aether-7B-5Attn-it
Aether-7B-5Attn-it
📚 Part of the Aether Foundation Model collection — base, instruction-tuned, and checkpoints in one place. Instruction-tuned (SFT) version of the fully-open Aether-7B-5Attn base model. Post-trained for multiple-choice / benchmark-style answering. 6.59B MoE (~2.98B active), 49 layers on a 7×7 Latin square. Apache-2.0.
Learning rate chosen by held-out accuracy, not loss
Full-parameter SFT was run at three learning rates (2e-6 / 6e-6 / 2e-5). The final checkpoint was selected by held-out benchmark accuracy, not training loss — for small models a high LR lowers training loss while degrading real capability, so loss is the wrong selector.
| LR | K-AI 4 avg | Note |
|---|---|---|
| 2e-6 | 29.7% | Under-fit (weak on some subjects) |
| 6e-6 (selected) | 34.9% | Best & balanced, no degradation |
| 2e-5 | 32.3% | One subject collapsed (format-degradation sign) |
Held-out results (selected checkpoint, 6e-6)
- GPQA-Diamond (198): 25.3%
-
K-AI 4 average (195):
34.9%
- musr_ko 26.5% · com2_main_ko 50.0% · click 32.0% · kommlu_pro 30.4%
- vs base (before SFT): base 26.7% → 34.9% (+8.2pp) — same held-out set, same harness.
All evaluations use held-out sets not seen in training. SFT was performed on MMLU-auxiliary (multiple-choice format), which does not overlap with the evaluation subjects (GPQA / K-AI).
Pretraining data mix (inherited from base)
| Domain | Share |
|---|---|
| Math (finemath + open-web-math) | 37.8% |
| Korean (webtext + synth) | 21.6% |
| English web & synthetic (fineweb-edu + cosmopedia) | 21.6% |
| Code (opc) | 13.5% |
| phase15 pre-blend | 5.4% |
This is the base pretraining mix. This model's post-training (SFT) data is MMLU-auxiliary.
Architecture (inherited from base)
Identical to Aether-7B-5Attn base: 49 layers placed on a 7×7 Latin square with heterogeneous attention (7 labels / 5 distinct mechanisms) and a 25-expert MoE (top-7 + 1 shared). Full structural detail and the diagram are in the base card, §3.2: FINAL-Bench/Aether-7B-5Attn .
Open-source fully-open LLMs — 6-country comparison
Relative to the base Aether. Among six sovereign fully-open models, VIDRAFT is the only single AI startup, and Aether has the most attention types (5) in a Latin-square layout.
Usage
Uses the custom
aether_pkg/
architecture (
trust_remote_code=True
). Follow the loading instructions in the
base model card
.
⚠️ Use batch_size=1. Some attention branches (NSA family) do not consume a padding mask, so batching padded sequences can silently corrupt results. Run inference one sequence at a time.
Contact
VIDRAFT (주식회사 비드래프트) · [email protected] · License: Apache-2.0
Runs of FINAL-Bench Aether-7B-5Attn-it on huggingface.co
168
Total runs
0
24-hour runs
-16
3-day runs
-28
7-day runs
-23
30-day runs
More Information About Aether-7B-5Attn-it huggingface.co Model
Url of Aether-7B-5Attn-it
Aether-7B-5Attn-it huggingface.co Url
Provider of Aether-7B-5Attn-it huggingface.co
Other API from FINAL-Bench
Updated:September 25 2026
Total runs:
271.4K
Run Growth:
-61.7K
Growth Rate:
-22.74%
Total runs:
90.5K
Run Growth:
90.5K
Growth Rate:
100.00%
Total runs:
6.0K
Run Growth:
3.7K
Growth Rate:
100.00%
Total runs:
5.8K
Run Growth:
5.7K
Growth Rate:
98.77%
Total runs:
2.5K
Run Growth:
2.5K
Growth Rate:
100.00%
Total runs:
2.2K
Run Growth:
1.9K
Growth Rate:
86.96%
Total runs:
2.1K
Run Growth:
2.1K
Growth Rate:
100.00%
Total runs:
1.5K
Run Growth:
1.5K
Growth Rate:
100.00%
Total runs:
1.5K
Run Growth:
1.4K
Growth Rate:
97.81%
Total runs:
1.2K
Run Growth:
1.1K
Growth Rate:
100.00%
Total runs:
995
Run Growth:
605
Growth Rate:
60.80%
Total runs:
988
Run Growth:
-61
Growth Rate:
-6.17%
Total runs:
912
Run Growth:
497
Growth Rate:
54.50%
Total runs:
806
Run Growth:
535
Growth Rate:
66.38%
Total runs:
600
Run Growth:
-122
Growth Rate:
-20.33%
Total runs:
550
Run Growth:
522
Growth Rate:
100.00%
Total runs:
491
Run Growth:
322
Growth Rate:
65.58%
Total runs:
471
Run Growth:
465
Growth Rate:
98.73%
Total runs:
432
Run Growth:
432
Growth Rate:
100.00%
Total runs:
401
Run Growth:
239
Growth Rate:
59.60%
Total runs:
385
Run Growth:
49
Growth Rate:
12.73%
Total runs:
378
Run Growth:
358
Growth Rate:
100.00%
Total runs:
346
Run Growth:
260
Growth Rate:
75.14%
Total runs:
305
Run Growth:
60
Growth Rate:
19.67%
Total runs:
256
Run Growth:
-18
Growth Rate:
-7.09%
Total runs:
254
Run Growth:
-84
Growth Rate:
-33.47%
Total runs:
234
Run Growth:
56
Growth Rate:
23.93%
Total runs:
228
Run Growth:
395
Growth Rate:
94.72%
Total runs:
191
Run Growth:
191
Growth Rate:
100.00%
Total runs:
162
Run Growth:
89
Growth Rate:
54.94%
Total runs:
157
Run Growth:
-55
Growth Rate:
-35.48%
Total runs:
156
Run Growth:
77
Growth Rate:
49.36%
Total runs:
153
Run Growth:
-82
Growth Rate:
-53.59%
Total runs:
127
Run Growth:
114
Growth Rate:
89.76%
Total runs:
119
Run Growth:
92
Growth Rate:
82.88%
Total runs:
117
Run Growth:
117
Growth Rate:
100.00%
Total runs:
116
Run Growth:
58
Growth Rate:
52.25%
Total runs:
109
Run Growth:
35
Growth Rate:
32.11%
Total runs:
108
Run Growth:
0
Growth Rate:
0.00%
Total runs:
72
Run Growth:
-180
Growth Rate:
-250.00%
Total runs:
63
Run Growth:
37
Growth Rate:
58.73%
Total runs:
61
Run Growth:
61
Growth Rate:
100.00%
Total runs:
60
Run Growth:
60
Growth Rate:
100.00%
Total runs:
60
Run Growth:
17
Growth Rate:
28.33%
Total runs:
52
Run Growth:
52
Growth Rate:
100.00%
Total runs:
52
Run Growth:
52
Growth Rate:
100.00%
Total runs:
51
Run Growth:
51
Growth Rate:
100.00%
Total runs:
50
Run Growth:
50
Growth Rate:
100.00%
Total runs:
48
Run Growth:
48
Growth Rate:
100.00%
Total runs:
47
Run Growth:
16
Growth Rate:
34.04%
Total runs:
46
Run Growth:
46
Growth Rate:
100.00%
Total runs:
46
Run Growth:
46
Growth Rate:
100.00%
Total runs:
43
Run Growth:
43
Growth Rate:
100.00%
Total runs:
40
Run Growth:
40
Growth Rate:
100.00%
Total runs:
31
Run Growth:
22
Growth Rate:
70.97%
Total runs:
13
Run Growth:
6
Growth Rate:
46.15%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%