Total runs:
11.7K
Run Growth:
1.5K
Growth Rate:
12.87%
Sponsored by
Skywork
utter-project / mHuBERT-147
Model's Last Updated: December 20 2024
feature-extraction
Introduction of mHuBERT-147
Model Details of mHuBERT-147
This repository contains the best mHuBERT-147 pre-trained model.
MODEL DETAILS: 3rd iteration, K=1000, HuBERT base architecture (95M parameters), 147 languages.
Table of Contents:
- Summary
- Training Data and Code
- ML-SUPERB Scores
- Languages and Datasets
- Citing and Funding Information
mHuBERT-147 models
mHuBERT-147 are compact and competitive multilingual HuBERT models trained on 90K hours of open-license data in 147 languages. Different from traditional HuBERTs, mHuBERT-147 models are trained using faiss IVF discrete speech units. Training employs a two-level language, data source up-sampling during training. See more information in our paper .
This repository contains:
- Fairseq checkpoint (original);
- HuggingFace checkpoint (conversion using transformers library);
- Faiss index for continuous pre-training (OPQ16_64,IVF1000_HNSW32,PQ16x4fsr).
Related Models:
Training
-
Manifest list available here. Please note that since training, there were CommonVoice removal requests. This means that some of the listed files are no longer available.
-
Fairseq fork contains the scripts for training with multilingual batching with two-level up-sampling.
ML-SUPERB Scores
mHubert-147 reaches second and first position in the 10min and 1h leaderboards respectively. We achieve new SOTA scores for three LID tasks. See more information in our paper .
Languages and Datasets
Datasets: For ASR/ST/TTS datasets, only train set is used.
- Aishell and AISHELL-3
- BibleTTS
- ClovaCall
- CommonVoice v11
- Google TTS data: Javanese , Khmer , Nepali , Sundanese , South African Languages , Bengali Languages
- IISc-MILE: Tamil , Kannada
- Japanese Versatile Speech
- Kokoro
- Kosp2e
- Media Speech: Turkish Only
- Multilingual LibriSpeech
- Samrómur
- THCHS-30 and THUYG-20
- VoxLingua107
- VoxPopuli
Languages present not indexed by Huggingface: Asturian (ast), Basaa (bas), Cebuano (ceb), Central Kurdish/Sorani (ckb), Hakha Chin (cnh), Hawaiian (haw), Upper Sorbian (hsb) Kabyle (kab), Moksha (mdf), Meadow Mari (mhr), Hill Mari (mrj), Erzya (myv), Taiwanese Hokkien (nan-tw), Sursilvan (rm-sursilv), Vallader (rm-vallader), Sakha (sah), Santali (sat), Scots (sco), Saraiki (skr), Tigre (tig), Tok Pisin (tpi), Akwapen Twi (tw-akuapem), Asante Twi (tw-asante), Votic (vot), Waray (war), Cantonese (yue).
Citing and Funding Information
@inproceedings{boito2024mhubert,
author={Marcely Zanon Boito, Vivek Iyer, Nikolaos Lagos, Laurent Besacier, Ioan Calapodescu},
title={{mHuBERT-147: A Compact Multilingual HuBERT Model}},
year=2024,
booktitle={Interspeech 2024},
}
This is an output of the European Project UTTER (Unified Transcription and Translation for Extended Reality) funded by European Union’s Horizon Europe Research and Innovation programme under grant agreement number 101070631.
For more information please visit https://he-utter.eu/
Runs of utter-project mHuBERT-147 on huggingface.co
51.2K
Total runs
0
24-hour runs
2.0K
3-day runs
2.3K
7-day runs
5.3K
30-day runs
More Information About mHuBERT-147 huggingface.co Model
Url of mHuBERT-147
mHuBERT-147 huggingface.co Url
Provider of mHuBERT-147 huggingface.co
Other API from utter-project
Updated:November 27 2024
Total runs:
10.8K
Run Growth:
-389
Growth Rate:
-3.61%
Total runs:
7.2K
Run Growth:
-11.7K
Growth Rate:
-161.21%
Total runs:
5.4K
Run Growth:
1.4K
Growth Rate:
26.65%
Total runs:
4.6K
Run Growth:
-7.6K
Growth Rate:
-163.58%
Total runs:
2.8K
Run Growth:
813
Growth Rate:
29.53%
Total runs:
2.7K
Run Growth:
-1.5K
Growth Rate:
-54.34%
Total runs:
2.5K
Run Growth:
1.1K
Growth Rate:
43.46%
Total runs:
1.2K
Run Growth:
-162
Growth Rate:
-13.80%
Total runs:
650
Run Growth:
395
Growth Rate:
60.77%
Total runs:
422
Run Growth:
-1000
Growth Rate:
-236.97%
Total runs:
178
Run Growth:
51
Growth Rate:
28.65%
Total runs:
147
Run Growth:
5
Growth Rate:
3.40%
Total runs:
80
Run Growth:
66
Growth Rate:
82.50%
Total runs:
78
Run Growth:
3
Growth Rate:
3.85%
Total runs:
55
Run Growth:
43
Growth Rate:
78.18%
Total runs:
51
Run Growth:
-580
Growth Rate:
-1137.25%
Total runs:
35
Run Growth:
6
Growth Rate:
17.14%
Total runs:
16
Run Growth:
10
Growth Rate:
62.50%
Total runs:
14
Run Growth:
-212
Growth Rate:
-1514.29%
Total runs:
11
Run Growth:
2
Growth Rate:
18.18%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
-107
Growth Rate:
-1188.89%
Total runs:
7
Run Growth:
-15
Growth Rate:
-214.29%
Total runs:
5
Run Growth:
-4
Growth Rate:
-80.00%
Total runs:
3
Run Growth:
-1
Growth Rate:
-33.33%
Total runs:
3
Run Growth:
-2
Growth Rate:
-66.67%
Total runs:
1
Run Growth:
-1
Growth Rate:
-100.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%