Sponsored by
APIMart
ufal / byt5-small-multilexnorm2021-da
Model's Last Updated: November 20 2023
text-generation
Introduction of byt5-small-multilexnorm2021-da
Model Details of byt5-small-multilexnorm2021-da
Fine-tuned ByT5-small for MultiLexNorm (Danish version)
This is the official release of the fine-tuned models for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task , which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5 , which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these fine-tuned models, we also release the source files on GitHub and an interactive demo on Google Colab .
How to use
The model was not fine-tuned in a standard sentence-to-sentence setting – instead, it was tailored to the token-to-token definition of MultiLexNorm data. Please refer to the interactive demo on Colab notebook to learn how to use these models.
How to cite
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
ByT5 - Small
ByT5 is a tokenizer-free version of Google's T5 and generally follows the architecture of MT5 .
ByT5 was only pre-trained on mC4 excluding any supervised training with an average span-mask of 20 UTF-8 characters. Therefore, this model has to be fine-tuned before it is useable on a downstream task.
ByT5 works especially well on noisy text data,
e.g.
,
google/byt5-small
significantly outperforms
mt5-small
on
TweetQA
.
Paper: ByT5: Towards a token-free future with pre-trained byte-to-byte models
Authors: Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
Runs of ufal byt5-small-multilexnorm2021-da on huggingface.co
8
Total runs
-1
24-hour runs
-2
3-day runs
-8
7-day runs
-17
30-day runs
More Information About byt5-small-multilexnorm2021-da huggingface.co Model
Url of byt5-small-multilexnorm2021-da
byt5-small-multilexnorm2021-da huggingface.co Url
Provider of byt5-small-multilexnorm2021-da huggingface.co
Other API from ufal
Total runs:
96
Run Growth:
-14
Growth Rate:
-13.86%
Total runs:
68
Run Growth:
-75
Growth Rate:
-104.17%
Total runs:
65
Run Growth:
37
Growth Rate:
56.92%
Total runs:
60
Run Growth:
30
Growth Rate:
49.18%
Total runs:
41
Run Growth:
-23
Growth Rate:
-56.10%
Total runs:
38
Run Growth:
21
Growth Rate:
56.76%
Total runs:
34
Run Growth:
6
Growth Rate:
18.18%
Total runs:
27
Run Growth:
2
Growth Rate:
7.41%
Total runs:
24
Run Growth:
0
Growth Rate:
0.00%
Total runs:
18
Run Growth:
-4
Growth Rate:
-22.22%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
-9
Growth Rate:
-52.94%
Total runs:
15
Run Growth:
-6
Growth Rate:
-40.00%
Total runs:
14
Run Growth:
-10
Growth Rate:
-71.43%
Total runs:
12
Run Growth:
-26
Growth Rate:
-236.36%
Total runs:
10
Run Growth:
-6
Growth Rate:
-60.00%
Total runs:
10
Run Growth:
-17
Growth Rate:
-170.00%
Total runs:
6
Run Growth:
-3
Growth Rate:
-50.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%