Sponsored by
APIMaster
ufal / byt5-small-multilexnorm2021-nl
Model's Last Updated: July 20 2023
text-generation
Introduction of byt5-small-multilexnorm2021-nl
Model Details of byt5-small-multilexnorm2021-nl
Fine-tuned ByT5-small for MultiLexNorm (Dutch version)
This is the official release of the fine-tuned models for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task , which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5 , which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these fine-tuned models, we also release the source files on GitHub and an interactive demo on Google Colab .
How to use
The model was not fine-tuned in a standard sentence-to-sentence setting – instead, it was tailored to the token-to-token definition of MultiLexNorm data. Please refer to the interactive demo on Colab notebook to learn how to use these models.
How to cite
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
ByT5 - Small
ByT5 is a tokenizer-free version of Google's T5 and generally follows the architecture of MT5 .
ByT5 was only pre-trained on mC4 excluding any supervised training with an average span-mask of 20 UTF-8 characters. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
ByT5 works especially well on noisy text data,
e.g.
,
google/byt5-small
significantly outperforms
mt5-small
on
TweetQA
.
Paper: ByT5: Towards a token-free future with pre-trained byte-to-byte models
Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
Runs of ufal byt5-small-multilexnorm2021-nl on huggingface.co
30
Total runs
0
24-hour runs
-1
3-day runs
-1
7-day runs
-28
30-day runs
More Information About byt5-small-multilexnorm2021-nl huggingface.co Model
Url of byt5-small-multilexnorm2021-nl
byt5-small-multilexnorm2021-nl huggingface.co Url
Provider of byt5-small-multilexnorm2021-nl huggingface.co
Other API from ufal
Total runs:
112
Run Growth:
20
Growth Rate:
17.86%
Total runs:
71
Run Growth:
48
Growth Rate:
67.61%
Total runs:
55
Run Growth:
-21
Growth Rate:
-38.18%
Total runs:
52
Run Growth:
-2
Growth Rate:
-3.85%
Total runs:
34
Run Growth:
-44
Growth Rate:
-129.41%
Total runs:
33
Run Growth:
-14
Growth Rate:
-42.42%
Total runs:
27
Run Growth:
6
Growth Rate:
22.22%
Total runs:
24
Run Growth:
1
Growth Rate:
4.17%
Total runs:
24
Run Growth:
9
Growth Rate:
37.50%
Total runs:
23
Run Growth:
-56
Growth Rate:
-243.48%
Total runs:
23
Run Growth:
-1
Growth Rate:
-4.35%
Total runs:
21
Run Growth:
3
Growth Rate:
14.29%
Total runs:
21
Run Growth:
8
Growth Rate:
38.10%
Total runs:
21
Run Growth:
-3
Growth Rate:
-14.29%
Total runs:
17
Run Growth:
-1
Growth Rate:
-5.88%
Total runs:
16
Run Growth:
-6
Growth Rate:
-37.50%
Total runs:
9
Run Growth:
4
Growth Rate:
50.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%