fgaim / tiroberta-tiald-multi-task

TiRoBERTa Fine-tuned for Multi-task Abusiveness, Sentiment, and Topic Classification

This model is a fine-tuned version of TiRoBERTa on the TiALD dataset.

Tigrinya Abusive Language Detection (TiALD) Dataset is a large-scale, multi-task benchmark dataset for abusive language detection in the Tigrinya language. It consists of 13,717 YouTube comments annotated for abusiveness , sentiment , and topic tasks. The dataset includes comments written in both the Ge’ez script and prevalent non-standard Latin transliterations to mirror real-world usage.

⚠️ The dataset contains explicit, obscene, and potentially hateful language. It should be used for research purposes only. ⚠️

This work accompanies the paper "A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings" .

Model Usage

from transformers import pipeline

tiald_multitask = pipeline("text-classification", model="fgaim/tiroberta-tiald-all-tasks", top_k=11)
tiald_multitask("<text-to-classify>")

Performance Metrics

This model achieves the following results on the TiALD test set:

"abusiveness_metrics": {
    "accuracy": 0.8611111111111112,
    "macro_f1": 0.8611109396431353,
    "macro_recall": 0.8611111111111112,
    "macro_precision": 0.8611128943846637,
    "weighted_f1": 0.8611109396431355,
    "weighted_recall": 0.8611111111111112,
    "weighted_precision": 0.8611128943846637
},
"topic_metrics": {
    "accuracy": 0.6155555555555555,
    "macro_f1": 0.5491185274678864,
    "macro_recall": 0.5143416011263588,
    "macro_precision": 0.7341640739780486,
    "weighted_f1": 0.5944096153417657,
    "weighted_recall": 0.6155555555555555,
    "weighted_precision": 0.6870800624645906
},
"sentiment_metrics": {
    "accuracy": 0.6533333333333333,
    "macro_f1": 0.5340845253007789,
    "macro_recall": 0.5410170159158625,
    "macro_precision": 0.534652401599494,
    "weighted_f1": 0.6620101614004723,
    "weighted_recall": 0.6533333333333333,
    "weighted_precision": 0.6750245466592532
}

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• optimizer: Adam (betas=0.9, 0.999, epsilon=1e-08)
• lr_scheduler_type: linear
• num_epochs: 7.0
• seed: 42

Intended Usage

The TiALD dataset and models designed to support:

• Research in abusive language detection in low-resource languages
• Context-aware abuse, sentiment, and topic modeling
• Multi-task and transfer learning with digraphic scripts
• Evaluation of multilingual and fine-tuned language models

Researchers and developers should avoid using this dataset for direct moderation or enforcement tasks without human oversight.

Ethical Considerations

• Sensitive content : Contains toxic and offensive language. Use for research purposes only.
• Cultural sensitivity : Abuse is context-dependent; annotations were made by native speakers to account for cultural nuance.
• Bias mitigation : Data sampling and annotation were carefully designed to minimize reinforcement of stereotypes.
• Privacy : All the source content for the dataset is publicly available on YouTube.
• Respect for expression : The dataset should not be used for automated censorship without human review.

This research received IRB approval (Ref: KH2022-133) and followed ethical data collection and annotation practices, including informed consent of annotators.

Citation

If you use this model or the TiALD dataset in your work, please cite:

@misc{gaim-etal-2025-tiald-benchmark,
  title         = {A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings},
  author        = {Fitsum Gaim and Hoyun Song and Huije Lee and Changgeon Ko and Eui Jun Hwang and Jong C. Park},
  year          = {2025},
  eprint        = {2505.12116},
  archiveprefix = {arXiv},
  primaryclass  = {cs.CL},
  url           = {https://arxiv.org/abs/2505.12116}
}

License

This dataset is released under the Creative Commons Attribution 4.0 International License (CC BY 4.0) .