Total runs:
18.6K
Run Growth:
115
Growth Rate:
0.62%
Sponsored by
APIMart
ytu-ce-cosmos / modernbert-tr-base
Model's Last Updated: July 19 2026
fill-mask
Introduction of modernbert-tr-base
Model Details of modernbert-tr-base
ModernBERT-TR
Release blog · Model collection
A 150M-parameter Turkish encoder with an 8,192-token context window.
Model
ModernBERT-TR is a masked-language encoder based on the ModernBERT architecture with alternating global and local attention.
| Parameters | 149.4M |
| Context length | 8,192 tokens |
| Layers | 22 |
| Hidden width | 768 |
| Attention heads | 12 |
| GLU feed-forward width | 1,152 |
| Tokenizer | Cased WordPiece, 50,008 tokens |
Results
We evaluate ModernBERT-TR with task-specific fine-tuning on 28 TabiBench tasks, the fixed five-seed TrGLUE protocol, and frozen-encoder linear probing with encoder-fast-eval .
ModernBERT-TR leads the frozen-encoder evaluation and ranks second on TabiBench and TrGLUE.
Mean rank across 47 tasks grouped by capability; lower is better. ModernBERT-TR leads classification, pairwise inference, and retrieval, and ties TabiBERT on acceptability.
Usage
from transformers import pipeline
unmask = pipeline(
"fill-mask",
model="ytu-ce-cosmos/modernbert-tr-base",
)
unmask("Türkiye'nin başkenti [MASK]'dır.")
Training
We pretrain from random initialization with packed sequences, dynamic masking, BF16, and StableAdamW. Our effective batch size is 1.05M tokens.
| Phase | Training |
|---|---|
| 1k pretraining |
200B tokens at length 1,024; 6B-token warmup to
5e-4
, then constant learning rate; masking 0.30
|
| 1k short decay |
20B-token continuation from the 1k endpoint; decay from
5e-4
to 0; masking 0.15
|
| 1k low-LR continuation |
Rollback to about 40B tokens; 100B tokens at
1e-4
, masking 0.10
|
| 1k decay | 1k decay soup from 1k low-LR continuation and 1k short decay |
| 8k context extension |
50B-token continuation from the 1k endpoint at length 8,192; constant
2e-4
; masking 0.30
|
| 8k terminal decay |
20B-token continuation on the 8k mix; decay from
2e-4
to 0; masking 0.10
|
The 1k pretraining mix is about 81.4% Turkish, 8.5% English, 10.0% code, and 0.1% parallel text. The 8k mix adds long-form Turkish PDF, legal, and long-document sources.
Revisions
| Revision | Contents |
|---|---|
main
|
ModernBERT-TR; 50% 1k decayed soup and 50% 8k terminal decay |
1k-no-decay
|
Direct 200B-token 1k pretraining endpoint |
1k-decay
|
50% 1k low-LR continuation and 50% 1k short decay |
8k-no-decay
|
Direct 50B-token 8k context-extension endpoint |
License
Apache-2.0.
Runs of ytu-ce-cosmos modernbert-tr-base on huggingface.co
420
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About modernbert-tr-base huggingface.co Model
Url of modernbert-tr-base
modernbert-tr-base huggingface.co Url
Provider of modernbert-tr-base huggingface.co
Other API from ytu-ce-cosmos
Updated:July 01 2026
Total runs:
17.6K
Run Growth:
4.6K
Growth Rate:
26.23%
Total runs:
12.0K
Run Growth:
11.1K
Growth Rate:
92.44%
Total runs:
4.1K
Run Growth:
1.9K
Growth Rate:
46.28%
Total runs:
1.7K
Run Growth:
-1.3K
Growth Rate:
-76.82%
Total runs:
1.4K
Run Growth:
406
Growth Rate:
29.85%
Total runs:
1.1K
Run Growth:
531
Growth Rate:
46.29%
Total runs:
794
Run Growth:
217
Growth Rate:
27.33%
Total runs:
649
Run Growth:
-242
Growth Rate:
-37.29%
Total runs:
588
Run Growth:
-8
Growth Rate:
-1.36%
Total runs:
563
Run Growth:
539
Growth Rate:
95.74%
Total runs:
563
Run Growth:
280
Growth Rate:
49.73%
Total runs:
493
Run Growth:
424
Growth Rate:
86.00%
Total runs:
471
Run Growth:
-242
Growth Rate:
-51.38%
Total runs:
463
Run Growth:
209
Growth Rate:
45.14%
Total runs:
374
Run Growth:
238
Growth Rate:
63.64%
Total runs:
226
Run Growth:
53
Growth Rate:
23.45%
Total runs:
205
Run Growth:
-41
Growth Rate:
-20.00%
Total runs:
199
Run Growth:
163
Growth Rate:
81.91%
Total runs:
175
Run Growth:
136
Growth Rate:
77.71%
Total runs:
172
Run Growth:
-9
Growth Rate:
-5.23%
Total runs:
159
Run Growth:
16
Growth Rate:
10.06%
Total runs:
149
Run Growth:
50
Growth Rate:
33.56%
Total runs:
123
Run Growth:
67
Growth Rate:
54.47%
Total runs:
116
Run Growth:
-297
Growth Rate:
-256.03%
Total runs:
100
Run Growth:
100
Growth Rate:
100.00%
Total runs:
80
Run Growth:
3
Growth Rate:
3.75%
Total runs:
79
Run Growth:
-9
Growth Rate:
-11.39%
Total runs:
76
Run Growth:
-2
Growth Rate:
-2.63%
Total runs:
69
Run Growth:
21
Growth Rate:
30.43%
Total runs:
55
Run Growth:
34
Growth Rate:
61.82%
Total runs:
47
Run Growth:
2
Growth Rate:
4.26%
Total runs:
43
Run Growth:
-58
Growth Rate:
-134.88%
Total runs:
34
Run Growth:
22
Growth Rate:
64.71%
Total runs:
33
Run Growth:
33
Growth Rate:
100.00%
Total runs:
32
Run Growth:
29
Growth Rate:
90.63%
Total runs:
30
Run Growth:
-108
Growth Rate:
-360.00%
Total runs:
28
Run Growth:
5
Growth Rate:
17.86%
Total runs:
26
Run Growth:
26
Growth Rate:
100.00%
Total runs:
24
Run Growth:
18
Growth Rate:
75.00%
Total runs:
24
Run Growth:
24
Growth Rate:
100.00%
Total runs:
21
Run Growth:
21
Growth Rate:
100.00%
Total runs:
19
Run Growth:
-31
Growth Rate:
-163.16%
Total runs:
13
Run Growth:
1
Growth Rate:
7.69%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%