Total runs:
31.4K
Run Growth:
5.0K
Growth Rate:
16.74%
Sponsored by
APIMart
BSC-LT / speech-salamandra-es-en
Model's Last Updated: July 15 2026
Introduction of speech-salamandra-es-en
Model Details of speech-salamandra-es-en
speech-salamandra-es-en Model Card
speech-salamandra-es-en is a speech-language model built on top of salamandra-7b-instruct . It accepts text and optional speech input and it can perform Automatic Speech Recognition (ASR) and Spoken Question Answering (SQA).
DISCLAIMER: This version of Salamandra can expose intermediate reasoning/transcription text. Use
skip_thinking=Truein the helper method, orprocessor.decode_response(..., skip_thinking=True), to display only the final response.
Table of contents
- Model Details
- Intended Use
- Hardware and Software
- How to use
- Training Data
- Evaluation
- Additional Information
Model Details
Description
speech-salamandra-es-en is a finetuned version of salamandra-7b-instruct coupled with the speech encoder from SeamlessM4T v2 . It has been trained to follow general text instructions and to emit an internal thinking sequence followed by the final answer.
Intended Use
The model is intended for both research and commercial use on speech/text instruction-following tasks.
Hardware and Software
Training Framework
The code used to train speech-salamandra-es-en is based on the Transformers library, and will be publicly available soon.
Compute Infrastructure
This model was trained on MareNostrum 5 , a pre-exascale EuroHPC supercomputer hosted and operated by Barcelona Supercomputing Center .
Training was conducted on 4 nodes, each with the following specifications:
- 4x Nvidia Hopper GPUs with 64GB HBM2 memory
- 2x Intel Sapphire Rapids 8460Y+ at 2.3Ghz and 32c each (64 cores)
- 4x NDR200 (BW per node 800Gb/s)
- 512 GB of Main memory (DDR5)
How to use
Requirements
To use this model, ensure you have the following Python packages installed:
transformers>=4.51.2,torch,torchaudio,numpy,soundfile
Load the model and processor directly:
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
model = AutoModelForCausalLM.from_pretrained(
"BSC-LT/speech-salamandra-es-en",
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
processor = AutoProcessor.from_pretrained(
"BSC-LT/speech-salamandra-es-en",
trust_remote_code=True,
)
The convenience method returns the generated text. Set
skip_thinking=True
to remove the leading reasoning/transcription span and keep only the final answer:
import soundfile as sf
audio_path = "/path/to/audio/example.wav"
audio, sr = sf.read(audio_path)
question = "Who is angry in the audio?"
response = model.generate_response(
processor,
instruction=question,
audio=[(audio, sr)],
max_new_tokens=512,
skip_thinking=True,
)
print(response)
To inspect the full thinking/transcription sequence, keep
skip_thinking=False
:
full_response = model.generate_response(
processor,
instruction=question,
audio=[(audio, sr)],
max_new_tokens=512,
skip_thinking=False,
)
print(full_response)
For an advanced usage, the
generate
method can be directly called:
inputs = processor.prepare_inputs(
instruction=question,
audio=[(audio, sr)],
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
input_length = inputs["input_ids"].shape[-1]
inputs["input_features"] = inputs["input_features"].unsqueeze(0).to(
model.device,
dtype=model.dtype,
)
inputs.pop("audio_attention_mask")
inputs["audio_lengths"] = inputs["audio_lengths"].to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
response = processor.decode_response(outputs, input_length=input_length, skip_thinking=True)
Training Data
Global Summary
| Data Type | Hours | Samples |
|---|---|---|
| ASR | 8289.5h | - |
| SQA | 2377h | - |
| IF | - | 327427 |
Automatic Speech Recognition (ASR)
| Dataset | en | es | Total |
|---|---|---|---|
| Common Voice Corpus 22.0 | 1762h | 494h | 2256h |
| VoxPopuli | 501h | 149h | 650h |
| Multilingual Librispeech | 4466.5h | 917h | 5383.5h |
| Total (hours) | 6729.5h | 1560h | 8289.5h |
Spoken Question Answering (SQA)
[NOTE] We refer to SQuADv2 as its subset of non-answerable questions. While we refer to SQuAD as the "answerable" subset in its first version.
We have expanded the SQuAD v2 dataset to speech synthesizing the context of each sample using Kyutai's Pocket TTS . To reduce the length of the audios, before synthesizing, we summarize the contexts with Gemma4 31B .
| Dataset | en | es | Total |
|---|---|---|---|
| LibriSQA | 727h | - | 727h |
| SQuAD | 576h | 525h | 1101h |
| SQuADv2 | 293h | 256h | 549h |
| Total (hours) | 1596h | 781h | 2377h |
Instruction Following Data
Although the model we are using as backbone is already insructed, we still add IF data during the training phase in order to avoid catastrophic forgetting. We use a subset of the original IF data used in the training of ALIA-40B-Instruct-2605 . This subset is composed of:
| Dataset | # of samples |
|---|---|
| WildChat | 153392 |
| Aya Dataset | 15146 |
| Mentor | 12719 |
| Other | 39141 |
| Total | 220398 |
Evaluation
Below are the evaluation ASR results on the Common Voice 22.0 and Voxpopuli test sets, and the SQA results on LibriSQA test set and a subset of the SQuAD dataset that we've expanded to synthetic audios. We use as a baseline Voxtral-Mini to compare our model. We report the following metrics:
Click to show metrics details
-
WER: jiwer implementation. Applied the basic normalization from whisper_normalizer before computing WER. -
BERTScore: bert_score implementation. We report F1 measure.
Automatic Speech Recognition
Common Voice 22.0 Test
Common Voice 22.0 Test
WER
| SpeechSalamandra | Voxtral | |
|---|---|---|
| en | 9.18 | 11.21 |
| es | 6.67 | 5.44 |
Spoken Question Answering
SQuAD
SQuAD
BERTScore
| SpeechSalamandra | Voxtral | |
|---|---|---|
| en | 0.65 | 0.68 |
| es | 0.58 | 0.63 |
Alternative SQuAD evaluation
For the SQA tasks, the answers include information that is not strictly required by the question. For example:
- Question: How many authors participated in the paper?
- Answer: The number of authors participating in the paper is 5.
- Target Answer: The paper has been written by 5 scientists.
Even though the meaning of both answers is the same, the words used are different. Actually, that is one of the reasons to use BertScore when evaluating this kind of tasks. One way to make this evaluation more robust is to post-edit the output to make the answer as concise as possible. After post-editing, the answers would be something like this:
- Question: How many authors participated in the paper?
- Answer: 5.
- Target Answer: 5.
This practice increases the metrics for both SpeechSalamandra and Voxtral, and make their comparison more focused on “what the model says” and not on “how the model says it”:
| SpeechSalamandra | Voxtral | |
|---|---|---|
| en | 0.78 | 0.82 |
| es | 0.76 | 0.75 |
Additional information
Authors
All the members of the Speech Salamandra Team are members of the AI Institute at the Barcelona Supercomputing Center (BSC) .
- Core Contributors (alphabetical order)
- Team Coordination
- Lead
Contact
For further information, please send an email to [email protected] .
Copyright
Copyright(c) 2026 by AI Institute, Barcelona Supercomputing Center.
Funding
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Modelos del Lenguaje.
Acknowledgements
The training of the model was possible thanks to the computing time provided by Barcelona Supercomputing Center through MareNostrum 5. We acknowledge EuroHPC Joint Undertaking for awarding us access to MareNostrum5 as BSC, Spain.
Disclaimer
Be aware that the model may contain biases or other unintended distortions. When third parties deploy systems or provide services based on this model, or use the model themselves, they bear the responsibility for mitigating any associated risks and ensuring compliance with applicable regulations, including those governing the use of Artificial Intelligence.
The Barcelona Supercomputing Center, as the owner and creator of the model, shall not be held liable for any outcomes resulting from third-party use.
License
Citation
If you find our model useful, we would appreciate if you could cite our work as follows:
@misc{bsclt2026speechsalamandra7b ,
title={speech-salamandra-es-en: a Speech LLM model for SQA in English and Spanish.},
author={Speech Salamandra Team},
organization={Barcelona Supercomputing Center},
url={https://huggingface.co/BSC-LT/speech-salamandra-es-en},
year={2026}
}
Runs of BSC-LT speech-salamandra-es-en on huggingface.co
73
Total runs
2
24-hour runs
4
3-day runs
13
7-day runs
57
30-day runs
More Information About speech-salamandra-es-en huggingface.co Model
Url of speech-salamandra-es-en
speech-salamandra-es-en huggingface.co Url
Provider of speech-salamandra-es-en huggingface.co
Other API from BSC-LT
Updated:October 22 2025
Total runs:
2.3K
Run Growth:
-865
Growth Rate:
-37.32%
Total runs:
1.7K
Run Growth:
-697
Growth Rate:
-40.83%
Total runs:
1.6K
Run Growth:
-3.6K
Growth Rate:
-229.90%
Total runs:
1.6K
Run Growth:
-1.7K
Growth Rate:
-116.93%
Total runs:
1.3K
Run Growth:
-157
Growth Rate:
-12.37%
Total runs:
814
Run Growth:
-163
Growth Rate:
-20.02%
Total runs:
692
Run Growth:
561
Growth Rate:
81.07%
Total runs:
567
Run Growth:
300
Growth Rate:
52.91%
Total runs:
503
Run Growth:
493
Growth Rate:
98.01%
Total runs:
472
Run Growth:
68
Growth Rate:
14.41%
Total runs:
453
Run Growth:
-690
Growth Rate:
-152.32%
Total runs:
354
Run Growth:
240
Growth Rate:
67.80%
Total runs:
336
Run Growth:
-743
Growth Rate:
-221.13%
Total runs:
242
Run Growth:
27
Growth Rate:
11.39%
Total runs:
238
Run Growth:
101
Growth Rate:
41.22%
Total runs:
234
Run Growth:
229
Growth Rate:
97.86%
Total runs:
206
Run Growth:
-2
Growth Rate:
-0.97%
Total runs:
203
Run Growth:
30
Growth Rate:
14.78%
Total runs:
180
Run Growth:
-4
Growth Rate:
-2.22%
Total runs:
174
Run Growth:
33
Growth Rate:
18.97%
Total runs:
147
Run Growth:
105
Growth Rate:
71.43%
Total runs:
146
Run Growth:
-131
Growth Rate:
-89.73%
Total runs:
145
Run Growth:
-161
Growth Rate:
-111.03%
Total runs:
142
Run Growth:
-46
Growth Rate:
-32.39%
Total runs:
141
Run Growth:
91
Growth Rate:
66.91%
Total runs:
137
Run Growth:
-666
Growth Rate:
-486.13%
Total runs:
134
Run Growth:
-14
Growth Rate:
-10.85%
Total runs:
131
Run Growth:
95
Growth Rate:
72.52%
Total runs:
127
Run Growth:
-58
Growth Rate:
-45.67%
Total runs:
124
Run Growth:
83
Growth Rate:
66.94%
Total runs:
120
Run Growth:
-85
Growth Rate:
-70.83%
Total runs:
119
Run Growth:
100
Growth Rate:
84.03%
Total runs:
117
Run Growth:
61
Growth Rate:
52.14%
Total runs:
113
Run Growth:
-134
Growth Rate:
-118.58%
Total runs:
109
Run Growth:
40
Growth Rate:
37.38%
Total runs:
107
Run Growth:
82
Growth Rate:
76.64%
Total runs:
106
Run Growth:
57
Growth Rate:
54.29%
Total runs:
106
Run Growth:
62
Growth Rate:
58.49%
Total runs:
105
Run Growth:
49
Growth Rate:
51.58%
Total runs:
103
Run Growth:
69
Growth Rate:
65.71%
Total runs:
102
Run Growth:
-23
Growth Rate:
-22.55%
Total runs:
102
Run Growth:
82
Growth Rate:
84.54%
Total runs:
97
Run Growth:
57
Growth Rate:
58.76%
Total runs:
97
Run Growth:
54
Growth Rate:
55.67%
Total runs:
92
Run Growth:
91
Growth Rate:
98.91%
Total runs:
91
Run Growth:
34
Growth Rate:
37.36%
Total runs:
86
Run Growth:
69
Growth Rate:
80.23%
Total runs:
86
Run Growth:
78
Growth Rate:
90.70%
Total runs:
84
Run Growth:
72
Growth Rate:
85.71%
Total runs:
79
Run Growth:
67
Growth Rate:
84.81%
Total runs:
73
Run Growth:
61
Growth Rate:
83.56%
Total runs:
51
Run Growth:
28
Growth Rate:
54.90%
Total runs:
49
Run Growth:
-25
Growth Rate:
-51.02%
Total runs:
47
Run Growth:
-57
Growth Rate:
-121.28%
Total runs:
47
Run Growth:
-6
Growth Rate:
-12.77%
Total runs:
45
Run Growth:
19
Growth Rate:
42.22%
Total runs:
43
Run Growth:
0
Growth Rate:
0.00%
Total runs:
40
Run Growth:
-18
Growth Rate:
-45.00%
Total runs:
39
Run Growth:
-6
Growth Rate:
-15.38%
Total runs:
36
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
-21
Growth Rate:
-67.74%
Total runs:
28
Run Growth:
15
Growth Rate:
53.57%
Total runs:
28
Run Growth:
-14
Growth Rate:
-50.00%
Total runs:
27
Run Growth:
-16
Growth Rate:
-59.26%
Total runs:
26
Run Growth:
-28
Growth Rate:
-107.69%
Total runs:
26
Run Growth:
-12
Growth Rate:
-46.15%
Total runs:
26
Run Growth:
-16
Growth Rate:
-61.54%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
21
Run Growth:
9
Growth Rate:
42.86%
Total runs:
19
Run Growth:
-201
Growth Rate:
-1057.89%
Total runs:
19
Run Growth:
5
Growth Rate:
26.32%
Total runs:
18
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
-14
Growth Rate:
-87.50%
Total runs:
15
Run Growth:
-3
Growth Rate:
-20.00%
Total runs:
11
Run Growth:
-1
Growth Rate:
-9.09%