SlEng-bert is a bilingual, Slovene-English masked language model.
SlEng-bert was trained from scratch on Slovene and English, conversational, non-standard, and slang language.
The model has 12 transformer layers, and is roughly equal in size to BERT and RoBERTa base models. The pre-training task used was masked language modeling, with no other tasks (like NSP).
The tokenizer and corpora used to train SlEng-bert were also used for training the
SloBERTa-SlEng
model.
The difference between the two is: SlEng-bert was trained from scratch for 40 epochs; SloBERTa-SlEng is SloBERTa further pre-trained for 2 epochs on new corpora.
Training corpora
The model was trained on English and Slovene tweets, Slovene corpora
MaCoCu
and
Frenk
,
and a small subset of English
Oscar
corpus. We tried to keep the sizes of English and Slovene corpora as equal as possible.
Training corpora had in total about 2.7 billion words.
Runs of cjvt sleng-bert on huggingface.co
11
Total runs
-1
24-hour runs
-2
3-day runs
-3
7-day runs
-26
30-day runs
More Information About sleng-bert huggingface.co Model
sleng-bert huggingface.co
sleng-bert huggingface.co is an AI model on huggingface.co that provides sleng-bert's model effect (), which can be used instantly with this cjvt sleng-bert model. huggingface.co supports a free trial of the sleng-bert model, and also provides paid use of the sleng-bert. Support call sleng-bert model through api, including Node.js, Python, http.
sleng-bert huggingface.co is an online trial and call api platform, which integrates sleng-bert's modeling effects, including api services, and provides a free online trial of sleng-bert, you can try sleng-bert online for free by clicking the link below.
cjvt sleng-bert online free url in huggingface.co:
sleng-bert is an open source model from GitHub that offers a free installation service, and any user can find sleng-bert on GitHub to install. At the same time, huggingface.co provides the effect of sleng-bert install, users can directly use sleng-bert installed effect in huggingface.co for debugging and trial. It also supports api for free installation.