Total runs:
2.6M
Run Growth:
2.6M
Growth Rate:
99.95%
Sponsored by
APIMart
speechbrain / m-ctc-t-large
Model's Last Updated: January 04 2023
automatic-speech-recognition
Introduction of m-ctc-t-large
Model Details of m-ctc-t-large
M-CTC-T
Massively multilingual speech recognizer from Meta AI. The model is a 1B-param transformer encoder, with a CTC head over 8065 character labels and a language identification head over 60 language ID labels. It is trained on Common Voice (version 6.1, December 2020 release) and VoxPopuli. After training on Common Voice and VoxPopuli, the model is trained on Common Voice only. The labels are unnormalized character-level transcripts (punctuation and capitalization are not removed). The model takes as input Mel filterbank features from a 16Khz audio signal.
The original Flashlight code, model checkpoints, and Colab notebook can be found at https://github.com/flashlight/wav2letter/tree/main/recipes/mling_pl .
Citation
Paper
Authors: Loren Lugosch, Tatiana Likhomanenko, Gabriel Synnaeve, Ronan Collobert
@article{lugosch2021pseudo,
title={Pseudo-Labeling for Massively Multilingual Speech Recognition},
author={Lugosch, Loren and Likhomanenko, Tatiana and Synnaeve, Gabriel and Collobert, Ronan},
journal={ICASSP},
year={2022}
}
Contribution
A huge thanks to Chan Woo Kim for porting the model from Flashlight C++ to PyTorch.
Training method
For more information on how the model was trained, please take a look at the
official paper
.
Usage
To transcribe audio files the model can be used as a standalone acoustic model as follows:
import torch
import torchaudio
from datasets import load_dataset
from transformers import MCTCTForCTC, MCTCTProcessor
model = MCTCTForCTC.from_pretrained("speechbrain/m-ctc-t-large")
processor = MCTCTProcessor.from_pretrained("speechbrain/m-ctc-t-large")
# load dummy dataset and read soundfiles
ds = load_dataset("patrickvonplaten/librispeech_asr_dummy", "clean", split="validation")
# feature extraction
input_features = processor(ds[0]["audio"]["array"], sampling_rate=ds[0]["audio"]["sampling_rate"], return_tensors="pt").input_features
# retrieve logits
with torch.no_grad():
logits = model(input_features).logits
# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)
Results for Common Voice, averaged over all languages:
Character error rate (CER) :
| "Valid" | "Test" |
|---|---|
| 21.4 | 23.3 |
Questions & Help
If you have questions regarding this model or need help, please consider opening a discussion or pull request on this repo and tag @lorenlugosch, @cwkeam or @patrickvonplaten
Runs of speechbrain m-ctc-t-large on huggingface.co
940
Total runs
4
24-hour runs
-23
3-day runs
-32
7-day runs
389
30-day runs
More Information About m-ctc-t-large huggingface.co Model
Url of m-ctc-t-large
m-ctc-t-large huggingface.co Url
Provider of m-ctc-t-large huggingface.co
Other API from speechbrain
Updated:February 26 2024
Total runs:
2.3M
Run Growth:
512.5K
Growth Rate:
22.55%
Total runs:
284.3K
Run Growth:
-29.4K
Growth Rate:
-10.36%
Total runs:
230.8K
Run Growth:
-24.5K
Growth Rate:
-10.62%
Total runs:
157.4K
Run Growth:
30.4K
Growth Rate:
18.97%
Total runs:
43.0K
Run Growth:
-7.4K
Growth Rate:
-17.69%
Total runs:
13.1K
Run Growth:
-3.8K
Growth Rate:
-29.39%
Total runs:
12.7K
Run Growth:
-9.6K
Growth Rate:
-75.54%
Total runs:
6.9K
Run Growth:
3.5K
Growth Rate:
49.21%
Total runs:
3.4K
Run Growth:
2.0K
Growth Rate:
56.65%
Total runs:
3.2K
Run Growth:
2.2K
Growth Rate:
64.32%
Total runs:
3.1K
Run Growth:
2.2K
Growth Rate:
67.63%
Total runs:
2.4K
Run Growth:
-218
Growth Rate:
-9.20%
Total runs:
1.3K
Run Growth:
-1.1K
Growth Rate:
-88.04%
Total runs:
1.1K
Run Growth:
-1.7K
Growth Rate:
-155.48%
Total runs:
1.0K
Run Growth:
747
Growth Rate:
72.31%
Total runs:
983
Run Growth:
79
Growth Rate:
8.04%
Total runs:
900
Run Growth:
-2.1K
Growth Rate:
-231.89%
Total runs:
794
Run Growth:
542
Growth Rate:
68.09%
Total runs:
659
Run Growth:
-383
Growth Rate:
-58.12%
Total runs:
525
Run Growth:
-215
Growth Rate:
-40.95%
Total runs:
493
Run Growth:
-70
Growth Rate:
-14.31%
Total runs:
437
Run Growth:
376
Growth Rate:
86.64%
Total runs:
312
Run Growth:
-41
Growth Rate:
-13.14%
Total runs:
287
Run Growth:
77
Growth Rate:
26.83%
Total runs:
277
Run Growth:
-10
Growth Rate:
-3.80%
Total runs:
267
Run Growth:
164
Growth Rate:
58.36%
Total runs:
259
Run Growth:
-608
Growth Rate:
-234.75%
Total runs:
245
Run Growth:
60
Growth Rate:
24.49%
Total runs:
217
Run Growth:
28
Growth Rate:
12.90%
Total runs:
211
Run Growth:
-34
Growth Rate:
-16.27%
Total runs:
194
Run Growth:
-190
Growth Rate:
-97.94%
Total runs:
186
Run Growth:
0
Growth Rate:
0.00%
Total runs:
182
Run Growth:
48
Growth Rate:
26.37%
Total runs:
174
Run Growth:
4
Growth Rate:
2.30%
Total runs:
155
Run Growth:
23
Growth Rate:
12.78%
Total runs:
136
Run Growth:
-77
Growth Rate:
-56.62%
Total runs:
121
Run Growth:
-32
Growth Rate:
-26.45%
Total runs:
98
Run Growth:
3
Growth Rate:
3.06%
Total runs:
95
Run Growth:
-97
Growth Rate:
-102.11%
Total runs:
83
Run Growth:
-52
Growth Rate:
-62.65%
Total runs:
83
Run Growth:
-267
Growth Rate:
-321.69%
Total runs:
66
Run Growth:
-123
Growth Rate:
-186.36%
Total runs:
64
Run Growth:
-61
Growth Rate:
-93.85%
Total runs:
63
Run Growth:
17
Growth Rate:
26.98%
Total runs:
61
Run Growth:
11
Growth Rate:
18.03%
Total runs:
60
Run Growth:
-113
Growth Rate:
-188.33%
Total runs:
59
Run Growth:
-53
Growth Rate:
-89.83%
Total runs:
58
Run Growth:
-84
Growth Rate:
-144.83%
Total runs:
57
Run Growth:
7
Growth Rate:
12.28%
Total runs:
53
Run Growth:
-150
Growth Rate:
-283.02%
Total runs:
52
Run Growth:
-37
Growth Rate:
-71.15%
Total runs:
52
Run Growth:
-49
Growth Rate:
-87.50%
Total runs:
52
Run Growth:
13
Growth Rate:
19.40%
Total runs:
48
Run Growth:
-22
Growth Rate:
-78.57%
Total runs:
47
Run Growth:
15
Growth Rate:
31.91%
Total runs:
47
Run Growth:
6
Growth Rate:
12.77%
Total runs:
44
Run Growth:
-149
Growth Rate:
-240.32%
Total runs:
42
Run Growth:
17
Growth Rate:
40.48%
Total runs:
41
Run Growth:
-15
Growth Rate:
-36.59%
Total runs:
39
Run Growth:
-176
Growth Rate:
-451.28%
Total runs:
39
Run Growth:
5
Growth Rate:
12.82%
Total runs:
36
Run Growth:
-158
Growth Rate:
-438.89%
Total runs:
34
Run Growth:
-5
Growth Rate:
-9.26%
Total runs:
33
Run Growth:
-110
Growth Rate:
-333.33%
Total runs:
33
Run Growth:
4
Growth Rate:
12.12%
Total runs:
32
Run Growth:
-7
Growth Rate:
-24.14%
Total runs:
32
Run Growth:
17
Growth Rate:
42.50%
Total runs:
30
Run Growth:
0
Growth Rate:
0.00%
Total runs:
28
Run Growth:
-37
Growth Rate:
-132.14%
Total runs:
28
Run Growth:
-25
Growth Rate:
-89.29%
Total runs:
26
Run Growth:
-60
Growth Rate:
-230.77%
Total runs:
25
Run Growth:
2
Growth Rate:
8.00%
Total runs:
25
Run Growth:
-42
Growth Rate:
-168.00%
Total runs:
25
Run Growth:
1
Growth Rate:
4.00%
Total runs:
24
Run Growth:
-16
Growth Rate:
-66.67%
Total runs:
24
Run Growth:
-1
Growth Rate:
-4.17%
Total runs:
23
Run Growth:
-53
Growth Rate:
-196.30%
Total runs:
23
Run Growth:
-43
Growth Rate:
-186.96%
Total runs:
22
Run Growth:
-33
Growth Rate:
-150.00%
Total runs:
21
Run Growth:
-16
Growth Rate:
-76.19%
Total runs:
21
Run Growth:
-26
Growth Rate:
-123.81%
Total runs:
21
Run Growth:
-16
Growth Rate:
-76.19%
Total runs:
20
Run Growth:
-28
Growth Rate:
-140.00%
Total runs:
20
Run Growth:
-25
Growth Rate:
-125.00%
Total runs:
20
Run Growth:
-2
Growth Rate:
-10.00%
Total runs:
20
Run Growth:
8
Growth Rate:
40.00%
Total runs:
20
Run Growth:
-9
Growth Rate:
-45.00%
Total runs:
18
Run Growth:
-31
Growth Rate:
-172.22%
Total runs:
18
Run Growth:
-34
Growth Rate:
-188.89%
Total runs:
16
Run Growth:
-9
Growth Rate:
-56.25%
Total runs:
16
Run Growth:
14
Growth Rate:
87.50%
Total runs:
15
Run Growth:
-29
Growth Rate:
-193.33%
Total runs:
15
Run Growth:
-1
Growth Rate:
-6.67%
Total runs:
14
Run Growth:
-25
Growth Rate:
-178.57%
Total runs:
14
Run Growth:
-149
Growth Rate:
-1064.29%
Total runs:
14
Run Growth:
-30
Growth Rate:
-214.29%
Total runs:
13
Run Growth:
7
Growth Rate:
53.85%
Total runs:
12
Run Growth:
-2
Growth Rate:
-16.67%
Total runs:
12
Run Growth:
10
Growth Rate:
83.33%