Sponsored by
Vmake Labs Video Enhancer
ufal / byt5-small-multilexnorm2021-it
Model's Last Updated: February 19 2025
text-generation
Introduction of byt5-small-multilexnorm2021-it
Model Details of byt5-small-multilexnorm2021-it
Fine-tuned ByT5-small for MultiLexNorm (Italian version)
This is the official release of the fine-tuned models for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task , which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5 , which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these fine-tuned models, we also release the source files on GitHub and an interactive demo on Google Colab .
How to use
The model was not fine-tuned in a standard sentence-to-sentence setting – instead, it was tailored to the token-to-token definition of MultiLexNorm data. Please refer to the interactive demo on Colab notebook to learn how to use these models.
How to cite
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
ByT5 - Small
ByT5 is a tokenizer-free version of Google's T5 and generally follows the architecture of MT5 .
ByT5 was only pre-trained on mC4 excluding any supervised training with an average span-mask of 20 UTF-8 characters. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
ByT5 works especially well on noisy text data,
e.g.
,
google/byt5-small
significantly outperforms
mt5-small
on
TweetQA
.
Paper: ByT5: Towards a token-free future with pre-trained byte-to-byte models
Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
Runs of ufal byt5-small-multilexnorm2021-it on huggingface.co
15
Total runs
0
24-hour runs
1
3-day runs
-4
7-day runs
-6
30-day runs
More Information About byt5-small-multilexnorm2021-it huggingface.co Model
Url of byt5-small-multilexnorm2021-it
byt5-small-multilexnorm2021-it huggingface.co Url
Provider of byt5-small-multilexnorm2021-it huggingface.co
Other API from ufal
Total runs:
96
Run Growth:
-14
Growth Rate:
-13.86%
Total runs:
68
Run Growth:
-75
Growth Rate:
-110.29%
Total runs:
68
Run Growth:
40
Growth Rate:
58.82%
Total runs:
60
Run Growth:
30
Growth Rate:
49.18%
Total runs:
41
Run Growth:
-23
Growth Rate:
-56.10%
Total runs:
38
Run Growth:
21
Growth Rate:
56.76%
Total runs:
34
Run Growth:
6
Growth Rate:
18.18%
Total runs:
27
Run Growth:
2
Growth Rate:
7.41%
Total runs:
23
Run Growth:
-1
Growth Rate:
-4.35%
Total runs:
19
Run Growth:
-3
Growth Rate:
-15.79%
Total runs:
17
Run Growth:
1
Growth Rate:
5.88%
Total runs:
17
Run Growth:
-9
Growth Rate:
-52.94%
Total runs:
15
Run Growth:
-9
Growth Rate:
-60.00%
Total runs:
12
Run Growth:
-25
Growth Rate:
-208.33%
Total runs:
11
Run Growth:
-5
Growth Rate:
-45.45%
Total runs:
10
Run Growth:
-17
Growth Rate:
-170.00%
Total runs:
8
Run Growth:
-17
Growth Rate:
-212.50%
Total runs:
6
Run Growth:
-3
Growth Rate:
-50.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%