Total runs:
28.6K
Run Growth:
6.2K
Growth Rate:
21.70%
Sponsored by
YouArt
BSC-LT / salamandra-7b-vision
Model's Last Updated: December 19 2025
image-text-to-text
Introduction of salamandra-7b-vision
Model Details of salamandra-7b-vision
Salamandra Vision Model Card
Salamandra is a highly multilingual model pre-trained from scratch that comes in three different sizes — 2B, 7B and 40B parameters — with their respective base and instruction-tuned variants. This model card corresponds to the 7B visual instructed version. Only the 7b model is currenlty instructed to understand images.
To visit the model cards of other Salamandra versions, please refer to the Model Index .
DISCLAIMER: This model is a first proof-of-concept designed to demonstrate the instruction-following capabilities of recently released base models. It has been optimized to engage in conversation but has NOT been aligned through RLHF to filter or avoid sensitive topics. As a result, it may generate harmful or inappropriate content. The team is actively working to enhance its performance through further instruction and alignment with RL techniques.
Model Details
Description
We have adapted Salamandra to process images and videos. This was achieved through late-fusion techniques, which involve integrating a pre-trained encoder, a pre-trained LLM, and a projector. The training process focuses on transforming the encoder's image embeddings to align with the LLM, enabling the model to comprehend this new modality.
Salamandra is a transformer-based decoder-only language model that has been pre-trained from scratch on 7.8 trillion tokens of highly curated data. The pre-training corpus contains text in 35 European languages and code.
Hyperparameters
The full list of hyperparameters can be found here .
Framework
We utilized the Llava Onevision technique to train our vision model.
The model comprises a pre-trained encoder ( Google SigLIP - 14 patches, 384x384 resolution), our 7B-instructed as the LLM, and a projector initialized from scratch (2-layer MLP).
Intended Use
Out-of-scope Use
The model is not intended for malicious activities, such as harming others or violating human rights. Any downstream application must comply with current laws and regulations. Irresponsible usage in production environments without proper risk assessment and mitigation is also discouraged.
Hardware and Software
Training Framework
The visual instruction-tuned versions were produced with Llava_Onevision .
Compute Infrastructure
All models were trained on MareNostrum 5 , a pre-exascale EuroHPC supercomputer hosted and operated by Barcelona Supercomputing Center.
The accelerated partition is composed of 1,120 nodes with the following specifications:
- 4x Nvidia Hopper GPUs with 64 HBM2 memory
- 2x Intel Sapphire Rapids 8460Y+ at 2.3Ghz and 32c each (64 cores)
- 4x NDR200 (BW per node 800Gb/s)
- 512 GB of Main memory (DDR5)
- 460GB on NVMe storage
How to use
from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
import torch
from PIL import Image
path = "BSC-LT/salamandra-7b-vision"
processor = AutoProcessor.from_pretrained(path)
model = LlavaOnevisionForConditionalGeneration.from_pretrained(path, torch_dtype=torch.float16, low_cpu_mem_usage=True).to("cuda")
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"
image = Image.open(url)
conversation = [
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": "Describe la imagen con el mayor detalle posible."},
],
},
]
prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
inputs = processor(images=image, text=prompt, return_tensors="pt").to("cuda", torch.float16)
output = model.generate(**inputs,
temperature=0.7,
max_new_tokens=1024)
output_tokens = output[0].tolist()
print(processor.decode(output[0], skip_special_tokens=True))
Using this template, each turn is preceded by a
<|im_start|>
delimiter and the role of the entity
(either
user
, for content supplied by the user, or
assistant
for LLM responses), and finished with the
<|im_end|>
token.
Data
The data distribution used to finetune the model is illustrated in the figure below . Most of them were sourced from LLaVA OneVision preprocessed data. This includes data from AI2D, Cambrian, and high-quality datasets such as re-captioned detailed description data from LLaVA Next. Diverse thematic data were included to enhance the model's capabilities in subtasks such as grounding, OCR, document understanding, and math. Additionally, we incorporated text-only multilingual data in various European languages and high-quality text-only data in Spanish, Catalan, Galician, and Basque, which were also used in the instruction tuning stage.
Evaluation
As there is a lack of multimodal multilingual evaluation data, we haven't performed a thorough multilingual evaluation yet (coming soon). The English evaluations are shown in the table below:
| Task | Subtask | Metric | Value |
|---|---|---|---|
| ai2d | exact_match | 0.7451 | |
| mme | cognition_score | mme_cognition_score | 246.4286 |
| perception_score | mme_perception_score | 1371.8164 | |
| mmmu_val | accuracy | 0.3689 | |
| mmstar | average | accuracy | 0.4865 |
| coarse perception | accuracy | 0.7127 | |
| fine-grained perception | accuracy | 0.3799 | |
| instance reasoning | accuracy | 0.5674 | |
| logical reasoning | accuracy | 0.4478 | |
| math | accuracy | 0.4279 | |
| science & technology | accuracy | 0.3832 | |
| realworldqa | exact_match | 0.5699 | |
| mmbench_en_dev | exact_match | 0.7113 | |
| docvqa_val | anls | 0.6805 | |
| infovqa_val | anls | 0.4859 |
Ethical Considerations and Limitations
This model is an initial prototype, and we have not yet conducted a thorough evaluation of societal and cognitive biases. In future iterations, we plan to assess potential biases using established benchmarks, following methodologies similar to those applied in previous models.
We acknowledge that bias evaluation is a critical step in responsible model development. Given the ongoing nature of this work, we strongly encourage developers to conduct safety assessments and bias mitigation strategies tailored to their specific applications of the model. Future updates will include more comprehensive analyses as we continue improving this model.
Additional information
Author
The Language Technologies Lab from Barcelona Supercomputing Center.
Contact
For further information, please send an email to [email protected] .
Copyright
Copyright(c) 2025 by Language Technologies Lab, Barcelona Supercomputing Center.
Funding
This work has been promoted and financed by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia - Funded by EU – NextGenerationEU within the framework of the project Modelos del Lenguaje.
Disclaimer
Be aware that the model may contain biases or other unintended distortions. When third parties deploy systems or provide services based on this model, or use the model themselves, they bear the responsibility for mitigating any associated risks and ensuring compliance with applicable regulations, including those governing the use of Artificial Intelligence.
The Barcelona Supercomputing Center, as the owner and creator of the model, shall not be held liable for any outcomes resulting from third-party use.
Citation
@misc{gonzalezagirre2025salamandratechnicalreport,
title={Salamandra Technical Report},
author={Aitor Gonzalez-Agirre and Marc Pàmies and Joan Llop and Irene Baucells and Severino Da Dalt and Daniel Tamayo and José Javier Saiz and Ferran Espuña and Jaume Prats and Javier Aula-Blasco and Mario Mina and Adrián Rubio and Alexander Shvets and Anna Sallés and Iñaki Lacunza and Iñigo Pikabea and Jorge Palomar and Júlia Falcão and Lucía Tormo and Luis Vasquez-Reina and Montserrat Marimon and Valle Ruíz-Fernández and Marta Villegas},
year={2025},
eprint={2502.08489},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.08489},
}
License
Base Model Index
References
- Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., & Li, C. (2024). Llava-OneVision: Easy Visual Task Transfer. [Link](https://arxiv.org/abs/2408.03326) - Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., & Lee, Y. J. (2024). Llava-Next: Improved Reasoning, OCR, and World Knowledge. [Link](https://llava-vl.github.io/blog/2024-01-30-llava-next/) - Kembhavi, A., Salvato, M., Kolve, E., Seo, M., Hajishirzi, H., & Farhadi, A. (2016). A Diagram is Worth a Dozen Images. [Link](https://arxiv.org/abs/1603.07396) - Tong, S., Brown, E., Wu, P., Woo, S., Middepogu, M., Akula, S. C., Yang, J., Yang, S., Iyer, A., Pan, X., Wang, Z., Fergus, R., LeCun, Y., & Xie, S. (2024). Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs. [Link](https://arxiv.org/abs/2406.16860) - - Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid Loss for Language Image Pre-Training. [Link](https://arxiv.org/abs/2303.15343)Runs of BSC-LT salamandra-7b-vision on huggingface.co
125
Total runs
0
24-hour runs
5
3-day runs
22
7-day runs
47
30-day runs
More Information About salamandra-7b-vision huggingface.co Model
Url of salamandra-7b-vision
salamandra-7b-vision huggingface.co Url
Provider of salamandra-7b-vision huggingface.co
Other API from BSC-LT
Updated:October 22 2025
Total runs:
2.3K
Run Growth:
-684
Growth Rate:
-29.21%
Total runs:
1.7K
Run Growth:
-533
Growth Rate:
-31.22%
Total runs:
1.3K
Run Growth:
-157
Growth Rate:
-12.37%
Total runs:
967
Run Growth:
-2.2K
Growth Rate:
-227.51%
Total runs:
769
Run Growth:
-830
Growth Rate:
-107.93%
Total runs:
676
Run Growth:
416
Growth Rate:
61.54%
Total runs:
598
Run Growth:
436
Growth Rate:
72.91%
Total runs:
503
Run Growth:
493
Growth Rate:
98.01%
Total runs:
468
Run Growth:
114
Growth Rate:
24.36%
Total runs:
467
Run Growth:
-619
Growth Rate:
-132.55%
Total runs:
346
Run Growth:
-691
Growth Rate:
-199.71%
Total runs:
240
Run Growth:
56
Growth Rate:
23.33%
Total runs:
234
Run Growth:
229
Growth Rate:
97.86%
Total runs:
203
Run Growth:
-18
Growth Rate:
-8.87%
Total runs:
200
Run Growth:
73
Growth Rate:
36.32%
Total runs:
186
Run Growth:
9
Growth Rate:
4.84%
Total runs:
176
Run Growth:
-50
Growth Rate:
-28.41%
Total runs:
174
Run Growth:
33
Growth Rate:
18.97%
Total runs:
162
Run Growth:
-65
Growth Rate:
-40.12%
Total runs:
146
Run Growth:
-131
Growth Rate:
-89.73%
Total runs:
145
Run Growth:
-161
Growth Rate:
-111.03%
Total runs:
142
Run Growth:
-46
Growth Rate:
-32.39%
Total runs:
138
Run Growth:
90
Growth Rate:
65.22%
Total runs:
134
Run Growth:
-32
Growth Rate:
-23.88%
Total runs:
133
Run Growth:
90
Growth Rate:
69.77%
Total runs:
130
Run Growth:
-26
Growth Rate:
-20.00%
Total runs:
126
Run Growth:
94
Growth Rate:
74.60%
Total runs:
116
Run Growth:
84
Growth Rate:
72.41%
Total runs:
116
Run Growth:
-103
Growth Rate:
-88.79%
Total runs:
111
Run Growth:
-24
Growth Rate:
-21.62%
Total runs:
111
Run Growth:
-512
Growth Rate:
-461.26%
Total runs:
104
Run Growth:
-217
Growth Rate:
-208.65%
Total runs:
102
Run Growth:
56
Growth Rate:
56.57%
Total runs:
97
Run Growth:
65
Growth Rate:
67.01%
Total runs:
97
Run Growth:
63
Growth Rate:
64.95%
Total runs:
94
Run Growth:
54
Growth Rate:
60.00%
Total runs:
92
Run Growth:
91
Growth Rate:
98.91%
Total runs:
91
Run Growth:
50
Growth Rate:
56.18%
Total runs:
87
Run Growth:
32
Growth Rate:
36.78%
Total runs:
87
Run Growth:
74
Growth Rate:
87.06%
Total runs:
86
Run Growth:
73
Growth Rate:
84.88%
Total runs:
85
Run Growth:
53
Growth Rate:
62.35%
Total runs:
82
Run Growth:
29
Growth Rate:
35.37%
Total runs:
80
Run Growth:
67
Growth Rate:
83.75%
Total runs:
79
Run Growth:
74
Growth Rate:
93.67%
Total runs:
76
Run Growth:
72
Growth Rate:
94.74%
Total runs:
75
Run Growth:
69
Growth Rate:
92.00%
Total runs:
72
Run Growth:
68
Growth Rate:
94.44%
Total runs:
68
Run Growth:
32
Growth Rate:
47.06%
Total runs:
49
Run Growth:
-25
Growth Rate:
-51.02%
Total runs:
44
Run Growth:
-34
Growth Rate:
-77.27%
Total runs:
43
Run Growth:
0
Growth Rate:
0.00%
Total runs:
39
Run Growth:
-6
Growth Rate:
-15.38%
Total runs:
36
Run Growth:
0
Growth Rate:
0.00%
Total runs:
34
Run Growth:
-25
Growth Rate:
-73.53%
Total runs:
31
Run Growth:
-80
Growth Rate:
-258.06%
Total runs:
30
Run Growth:
-11
Growth Rate:
-36.67%
Total runs:
29
Run Growth:
-2
Growth Rate:
-6.90%
Total runs:
26
Run Growth:
-8
Growth Rate:
-30.77%
Total runs:
26
Run Growth:
-6
Growth Rate:
-23.08%
Total runs:
25
Run Growth:
-16
Growth Rate:
-64.00%
Total runs:
25
Run Growth:
-6
Growth Rate:
-24.00%
Total runs:
25
Run Growth:
3
Growth Rate:
12.00%
Total runs:
23
Run Growth:
5
Growth Rate:
21.74%
Total runs:
23
Run Growth:
-1
Growth Rate:
-4.35%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
19
Run Growth:
-201
Growth Rate:
-1057.89%
Total runs:
19
Run Growth:
4
Growth Rate:
21.05%
Total runs:
18
Run Growth:
-9
Growth Rate:
-50.00%
Total runs:
16
Run Growth:
-14
Growth Rate:
-87.50%
Total runs:
15
Run Growth:
-3
Growth Rate:
-20.00%
Total runs:
11
Run Growth:
-1
Growth Rate:
-9.09%
Total runs:
11
Run Growth:
7
Growth Rate:
63.64%