Total runs:
16.6K
Run Growth:
13.9K
Growth Rate:
83.84%
Sponsored by
Agentkit
dicta-il / neodictabert
Model's Last Updated: October 27 2025
Introduction of neodictabert
Model Details of neodictabert
NeoDictaBERT: Pushing the Frontier of BERT models in Hebrew
Following the success of ModernBERT and NeoBERT , we set out to train a Hebrew version of NeoBERT.
Introducing NeoDictaBERT : A Next-Generation BERT-style model trained specifically for Hebrew, technical report coming soon.
Supported Context Length: 4,096 (~ 3,200 words)
Trained on a total of 235B tokens (5 epochs) with a context length of 1,024, and another 50B tokens with a context length of 4,096.
This is the base model pretrained with the masked-language-modeling objective.
Sample usage:
from transformers import AutoModelForMaskedLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('dicta-il/neodictabert')
model = AutoModelForMaskedLM.from_pretrained('dicta-il/neodictabert', trust_remote_code=True)
model.eval()
sentence = 'בשנת 1948 השלים אפרים קישון את [MASK] בפיסול מתכת ובתולדות האמנות והחל לפרסם מאמרים הומוריסטיים'
output = model(tokenizer.encode(sentence, return_tensors='pt'))
# the [MASK] is the 7th token (including [CLS])
import torch
top_2 = torch.topk(output.logits[0, 7, :], 2)[1]
print('\n'.join(tokenizer.convert_ids_to_tokens(top_2))) # should print לימודיו / הכשרתו
Performance
| Model | HeQ EM | HeQ F1 | HeQ TInls | Syntax-HTBv2=>HTBv2 UAS | Syntax-HTBv2=>HTBv2 LAS | Sentiment Acc | NER (NEMO) F1 | Segmentation Acc |
|---|---|---|---|---|---|---|---|---|
| mBERT | 69.08 | 74.32 | 77.01 | -- | -- | 84.21 | 79.11 | -- |
| AlephBertGimmel | 61.77 | 67.97 | 71.35 | -- | -- | 89.51 | 85.26 | -- |
| DictaBERT | 70.35 | 77.04 | 80.34 | 90.52 | 86.54 | 89.79 | 87.01 | 99.1 |
| NeoDictaBERT_1024 | 75.07 | 82.20 | 85.51 | 91.48 | 87.21 | -- | -- | -- |
| NeoDictaBERT_4096 | 76.40 | 82.83 | 85.86 | 91.77 | 87.80 | 89.61 | 86.97 | 99.2 |
| Δ vs DictaBERT | +6.05 | +5.79 | +5.52 | +1.25 | +1.26 | -0.18 | -0.04 | +0.1 |
Citation
If you use NeoDictaBERT in your research, please cite
NeoDictaBERT: Pushing the Frontier of BERT models in Hebrew
BibTeX:
@misc{NeoDictaBERT,
title = {NeoDictaBERT: Pushing the Frontier of BERT models in Hebrew},
author = {Shaltiel Shmidman and Avi Shmidman and Moshe Koppel},
howpublished = {Hugging Face},
url = {https://huggingface.co/dicta-il/neodictabert},
license = {CC BY 4.0},
note = {Technical report coming soon}
}
License
Shield:
This work is licensed under a Creative Commons Attribution 4.0 International License .
Runs of dicta-il neodictabert on huggingface.co
13.9K
Total runs
1.5K
24-hour runs
1.5K
3-day runs
2.2K
7-day runs
9.4K
30-day runs
More Information About neodictabert huggingface.co Model
Url of neodictabert
neodictabert huggingface.co Url
Provider of neodictabert huggingface.co
Other API from dicta-il
Updated:March 20 2024
Total runs:
16.1K
Run Growth:
6.8K
Growth Rate:
42.31%
Total runs:
14.6K
Run Growth:
9.0K
Growth Rate:
61.77%
Total runs:
6.2K
Run Growth:
2.0K
Growth Rate:
32.44%
Total runs:
5.2K
Run Growth:
1.7K
Growth Rate:
31.88%
Total runs:
4.5K
Run Growth:
182
Growth Rate:
4.07%
Total runs:
4.5K
Run Growth:
2.5K
Growth Rate:
55.61%
Total runs:
2.2K
Run Growth:
378
Growth Rate:
17.48%
Total runs:
1.9K
Run Growth:
-27
Growth Rate:
-1.42%
Total runs:
1.9K
Run Growth:
395
Growth Rate:
20.76%
Total runs:
1.8K
Run Growth:
-1.5K
Growth Rate:
-82.78%
Total runs:
1.4K
Run Growth:
-1.4K
Growth Rate:
-102.60%
Total runs:
1.3K
Run Growth:
-924
Growth Rate:
-74.10%
Total runs:
999
Run Growth:
38
Growth Rate:
3.80%
Total runs:
986
Run Growth:
165
Growth Rate:
18.33%
Total runs:
898
Run Growth:
815
Growth Rate:
90.76%
Total runs:
538
Run Growth:
-154
Growth Rate:
-28.62%
Total runs:
455
Run Growth:
-631
Growth Rate:
-138.68%
Total runs:
398
Run Growth:
-616
Growth Rate:
-154.77%
Total runs:
389
Run Growth:
-812
Growth Rate:
-208.74%
Total runs:
385
Run Growth:
-52
Growth Rate:
-13.51%
Total runs:
361
Run Growth:
-498
Growth Rate:
-137.95%
Total runs:
345
Run Growth:
-82
Growth Rate:
-23.77%
Total runs:
319
Run Growth:
292
Growth Rate:
91.54%
Total runs:
271
Run Growth:
130
Growth Rate:
47.97%
Total runs:
243
Run Growth:
-166
Growth Rate:
-68.31%
Total runs:
149
Run Growth:
-495
Growth Rate:
-332.21%
Total runs:
134
Run Growth:
0
Growth Rate:
0.00%
Total runs:
107
Run Growth:
-91
Growth Rate:
-85.05%
Total runs:
56
Run Growth:
-60
Growth Rate:
-107.14%
Total runs:
53
Run Growth:
-466
Growth Rate:
-879.25%
Total runs:
45
Run Growth:
36
Growth Rate:
80.00%
Total runs:
43
Run Growth:
-73
Growth Rate:
-169.77%
Total runs:
40
Run Growth:
28
Growth Rate:
70.00%
Total runs:
34
Run Growth:
15
Growth Rate:
44.12%
Total runs:
26
Run Growth:
17
Growth Rate:
65.38%
Total runs:
25
Run Growth:
-117
Growth Rate:
-468.00%
Total runs:
18
Run Growth:
6
Growth Rate:
33.33%
Total runs:
18
Run Growth:
1
Growth Rate:
5.56%
Total runs:
17
Run Growth:
11
Growth Rate:
64.71%
Total runs:
16
Run Growth:
-1
Growth Rate:
-6.25%
Total runs:
14
Run Growth:
6
Growth Rate:
42.86%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
1
Growth Rate:
11.11%
Total runs:
9
Run Growth:
-2
Growth Rate:
-22.22%
Total runs:
7
Run Growth:
-5
Growth Rate:
-71.43%
Total runs:
6
Run Growth:
-1
Growth Rate:
-16.67%