Sponsored by
Vmake Labs Video Enhancer
ufal / byt5-small-multilexnorm2021-sr
Model's Last Updated: February 19 2025
text-generation
Introduction of byt5-small-multilexnorm2021-sr
Model Details of byt5-small-multilexnorm2021-sr
Fine-tuned ByT5-small for MultiLexNorm (Serbian version)
This is the official release of the fine-tuned models for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task , which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5 , which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these fine-tuned models, we also release the source files on GitHub and an interactive demo on Google Colab .
How to use
The model was not fine-tuned in a standard sentence-to-sentence setting – instead, it was tailored to the token-to-token definition of MultiLexNorm data. Please refer to the interactive demo on Colab notebook to learn how to use these models.
How to cite
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
ByT5 - Small
ByT5 is a tokenizer-free version of Google's T5 and generally follows the architecture of MT5 .
ByT5 was only pre-trained on mC4 excluding any supervised training with an average span-mask of 20 UTF-8 characters. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
ByT5 works especially well on noisy text data,
e.g.
,
google/byt5-small
significantly outperforms
mt5-small
on
TweetQA
.
Paper: ByT5: Towards a token-free future with pre-trained byte-to-byte models
Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
Runs of ufal byt5-small-multilexnorm2021-sr on huggingface.co
15
Total runs
1
24-hour runs
0
3-day runs
-1
7-day runs
1
30-day runs
More Information About byt5-small-multilexnorm2021-sr huggingface.co Model
Url of byt5-small-multilexnorm2021-sr
byt5-small-multilexnorm2021-sr huggingface.co Url
Provider of byt5-small-multilexnorm2021-sr huggingface.co
Other API from ufal
Total runs:
105
Run Growth:
-5
Growth Rate:
-4.59%
Total runs:
63
Run Growth:
22
Growth Rate:
34.92%
Total runs:
56
Run Growth:
26
Growth Rate:
47.27%
Total runs:
40
Run Growth:
-26
Growth Rate:
-65.00%
Total runs:
37
Run Growth:
23
Growth Rate:
62.16%
Total runs:
33
Run Growth:
-14
Growth Rate:
-42.42%
Total runs:
33
Run Growth:
-33
Growth Rate:
-100.00%
Total runs:
29
Run Growth:
8
Growth Rate:
27.59%
Total runs:
23
Run Growth:
7
Growth Rate:
30.43%
Total runs:
23
Run Growth:
7
Growth Rate:
30.43%
Total runs:
23
Run Growth:
-14
Growth Rate:
-60.87%
Total runs:
21
Run Growth:
4
Growth Rate:
19.05%
Total runs:
20
Run Growth:
2
Growth Rate:
10.00%
Total runs:
17
Run Growth:
-15
Growth Rate:
-88.24%
Total runs:
16
Run Growth:
-2
Growth Rate:
-12.50%
Total runs:
15
Run Growth:
-7
Growth Rate:
-46.67%
Total runs:
12
Run Growth:
9
Growth Rate:
75.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%