Sponsored by
APIMart
ufal / byt5-small-multilexnorm2021-de
Model's Last Updated: February 19 2025
text-generation
Introduction of byt5-small-multilexnorm2021-de
Model Details of byt5-small-multilexnorm2021-de
Fine-tuned ByT5-small for MultiLexNorm (German version)
This is the official release of the fine-tuned models for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task , which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5 , which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these fine-tuned models, we also release the source files on GitHub and an interactive demo on Google Colab .
How to use
The model was not fine-tuned in a standard sentence-to-sentence setting – instead, it was tailored to the token-to-token definition of MultiLexNorm data. Please refer to the interactive demo on Colab notebook to learn how to use these models.
How to cite
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
ByT5 - Small
ByT5 is a tokenizer-free version of Google's T5 and generally follows the architecture of MT5 .
ByT5 was only pre-trained on mC4 excluding any supervised training with an average span-mask of 20 UTF-8 characters. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
ByT5 works especially well on noisy text data,
e.g.
,
google/byt5-small
significantly outperforms
mt5-small
on
TweetQA
.
Paper: ByT5: Towards a token-free future with pre-trained byte-to-byte models
Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
Runs of ufal byt5-small-multilexnorm2021-de on huggingface.co
34
Total runs
0
24-hour runs
1
3-day runs
-1
7-day runs
-13
30-day runs
More Information About byt5-small-multilexnorm2021-de huggingface.co Model
Url of byt5-small-multilexnorm2021-de
byt5-small-multilexnorm2021-de huggingface.co Url
Provider of byt5-small-multilexnorm2021-de huggingface.co
Other API from ufal
Total runs:
116
Run Growth:
22
Growth Rate:
19.30%
Total runs:
71
Run Growth:
47
Growth Rate:
66.20%
Total runs:
61
Run Growth:
-9
Growth Rate:
-14.75%
Total runs:
61
Run Growth:
10
Growth Rate:
16.13%
Total runs:
35
Run Growth:
-28
Growth Rate:
-77.78%
Total runs:
30
Run Growth:
14
Growth Rate:
48.28%
Total runs:
29
Run Growth:
-29
Growth Rate:
-100.00%
Total runs:
28
Run Growth:
7
Growth Rate:
25.00%
Total runs:
23
Run Growth:
5
Growth Rate:
21.74%
Total runs:
23
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
-57
Growth Rate:
-259.09%
Total runs:
22
Run Growth:
9
Growth Rate:
40.91%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
21
Run Growth:
-3
Growth Rate:
-14.29%
Total runs:
16
Run Growth:
-6
Growth Rate:
-37.50%
Total runs:
16
Run Growth:
-2
Growth Rate:
-12.50%
Total runs:
11
Run Growth:
5
Growth Rate:
55.56%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%