Total runs:
28.1K
Run Growth:
4.5K
Growth Rate:
15.60%
Sponsored by
Agentkit
BSC-LT / salamandra-2b-instruct-fp8
Model's Last Updated: March 26 2025
text-generation
Introduction of salamandra-2b-instruct-fp8
Model Details of salamandra-2b-instruct-fp8
Salamandra-2b-instruct-fp8 Model Card
This model is the fp8-quantized version of Salamandra-2b-instruct .
The model weights are quantized from FP16 to FP8 (8-bit weights) using the FP8 quantization algorithm from NeuralMagic . Inferencing with this model can be done using VLLM .
Salamandra is a highly multilingual model pre-trained from scratch that comes in three different sizes — 2B, 7B and 40B parameters — with their respective base and instruction-tuned variants, promoted and financed by the Government of Catalonia through the Aina Project and the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of ILENIA Project with reference 2022/TL22/00215337.
This model card corresponds to the fp8-quantized version of Salamandra-2b-instruct.
The entire Salamandra family is released under a permissive Apache 2.0 license .
How to Use
The following example code works under
Python 3.9.16
,
vllm==0.6.3.post1
,
torch==2.4.0
and
torchvision==0.19.0
, though it should run on
any current version of the libraries. This is an example of a conversational chatbot using the model:
from vllm import LLM, SamplingParams
model_name = "BSC-LT/salamandra-2b-instruct-fp8"
llm = LLM(model=model_name)
messages = []
while True:
user_input = input("user >> ")
if user_input.lower() == "exit":
print("Chat ended.")
break
messages.append({'role': 'user', 'content': user_input})
outputs = llm.chat(messages,
sampling_params=SamplingParams(
temperature=0.5,
stop_token_ids=[5],
max_tokens=200)
)[0].outputs
model_output = outputs[0].text
print(f'assistant >> {model_output}')
messages.append({'role': 'assistant', 'content': model_output})
Author
International Business Machines (IBM).
Copyright
International Business Machines (IBM).
Contact
For further information, please send an email to [email protected] .
Acknowledgements
We appreciate the collaboration with IBM in this work. Specifically, the IBM team created fp8-quantized version of the Salamandra-2b-instruct model released here.
Disclaimer
Be aware that the model may contain biases or other unintended distortions. When third parties deploy systems or provide services based on this model, or use the model themselves, they bear the responsibility for mitigating any associated risks and ensuring compliance with applicable regulations, including those governing the use of Artificial Intelligence.
Barcelona Supercomputing Center and International Business Machines shall not be held liable for any outcomes resulting from third-party use.
License
Runs of BSC-LT salamandra-2b-instruct-fp8 on huggingface.co
26
Total runs
0
24-hour runs
-8
3-day runs
-6
7-day runs
-18
30-day runs
More Information About salamandra-2b-instruct-fp8 huggingface.co Model
Url of salamandra-2b-instruct-fp8
salamandra-2b-instruct-fp8 huggingface.co Url
Provider of salamandra-2b-instruct-fp8 huggingface.co
Other API from BSC-LT
Updated:October 22 2025
Total runs:
2.3K
Run Growth:
-762
Growth Rate:
-33.13%
Total runs:
1.9K
Run Growth:
-2.1K
Growth Rate:
-112.79%
Total runs:
1.7K
Run Growth:
-743
Growth Rate:
-43.02%
Total runs:
1.3K
Run Growth:
-157
Growth Rate:
-12.37%
Total runs:
804
Run Growth:
-2.3K
Growth Rate:
-236.27%
Total runs:
798
Run Growth:
-753
Growth Rate:
-94.36%
Total runs:
713
Run Growth:
547
Growth Rate:
76.72%
Total runs:
681
Run Growth:
568
Growth Rate:
83.41%
Total runs:
503
Run Growth:
493
Growth Rate:
98.01%
Total runs:
498
Run Growth:
129
Growth Rate:
25.90%
Total runs:
475
Run Growth:
-652
Growth Rate:
-137.26%
Total runs:
350
Run Growth:
-707
Growth Rate:
-202.00%
Total runs:
240
Run Growth:
47
Growth Rate:
19.58%
Total runs:
234
Run Growth:
229
Growth Rate:
97.86%
Total runs:
228
Run Growth:
56
Growth Rate:
28.00%
Total runs:
220
Run Growth:
100
Growth Rate:
45.45%
Total runs:
184
Run Growth:
9
Growth Rate:
4.89%
Total runs:
182
Run Growth:
-37
Growth Rate:
-20.33%
Total runs:
174
Run Growth:
33
Growth Rate:
18.97%
Total runs:
171
Run Growth:
-53
Growth Rate:
-30.99%
Total runs:
146
Run Growth:
-131
Growth Rate:
-89.73%
Total runs:
145
Run Growth:
-161
Growth Rate:
-111.03%
Total runs:
142
Run Growth:
101
Growth Rate:
71.13%
Total runs:
142
Run Growth:
-46
Growth Rate:
-32.39%
Total runs:
135
Run Growth:
-45
Growth Rate:
-33.33%
Total runs:
133
Run Growth:
93
Growth Rate:
69.92%
Total runs:
131
Run Growth:
-63
Growth Rate:
-48.09%
Total runs:
130
Run Growth:
98
Growth Rate:
75.38%
Total runs:
126
Run Growth:
-632
Growth Rate:
-501.59%
Total runs:
122
Run Growth:
86
Growth Rate:
70.49%
Total runs:
117
Run Growth:
60
Growth Rate:
51.28%
Total runs:
112
Run Growth:
-26
Growth Rate:
-23.21%
Total runs:
105
Run Growth:
-177
Growth Rate:
-168.57%
Total runs:
104
Run Growth:
90
Growth Rate:
86.54%
Total runs:
102
Run Growth:
59
Growth Rate:
57.84%
Total runs:
101
Run Growth:
67
Growth Rate:
66.34%
Total runs:
101
Run Growth:
61
Growth Rate:
60.40%
Total runs:
100
Run Growth:
42
Growth Rate:
42.00%
Total runs:
96
Run Growth:
13
Growth Rate:
13.54%
Total runs:
94
Run Growth:
58
Growth Rate:
61.70%
Total runs:
92
Run Growth:
91
Growth Rate:
98.91%
Total runs:
92
Run Growth:
52
Growth Rate:
57.14%
Total runs:
90
Run Growth:
55
Growth Rate:
61.80%
Total runs:
89
Run Growth:
76
Growth Rate:
87.36%
Total runs:
84
Run Growth:
27
Growth Rate:
32.14%
Total runs:
81
Run Growth:
68
Growth Rate:
83.95%
Total runs:
80
Run Growth:
73
Growth Rate:
91.25%
Total runs:
77
Run Growth:
72
Growth Rate:
93.51%
Total runs:
76
Run Growth:
70
Growth Rate:
92.11%
Total runs:
74
Run Growth:
67
Growth Rate:
90.54%
Total runs:
69
Run Growth:
30
Growth Rate:
43.48%
Total runs:
49
Run Growth:
27
Growth Rate:
55.10%
Total runs:
49
Run Growth:
-25
Growth Rate:
-51.02%
Total runs:
43
Run Growth:
0
Growth Rate:
0.00%
Total runs:
40
Run Growth:
-33
Growth Rate:
-82.50%
Total runs:
39
Run Growth:
-18
Growth Rate:
-46.15%
Total runs:
39
Run Growth:
-6
Growth Rate:
-15.38%
Total runs:
37
Run Growth:
-68
Growth Rate:
-183.78%
Total runs:
36
Run Growth:
0
Growth Rate:
0.00%
Total runs:
30
Run Growth:
-22
Growth Rate:
-73.33%
Total runs:
28
Run Growth:
-14
Growth Rate:
-50.00%
Total runs:
26
Run Growth:
-16
Growth Rate:
-61.54%
Total runs:
25
Run Growth:
-15
Growth Rate:
-60.00%
Total runs:
25
Run Growth:
-29
Growth Rate:
-116.00%
Total runs:
25
Run Growth:
-2
Growth Rate:
-8.00%
Total runs:
23
Run Growth:
6
Growth Rate:
26.09%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
19
Run Growth:
-201
Growth Rate:
-1057.89%
Total runs:
19
Run Growth:
7
Growth Rate:
36.84%
Total runs:
18
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
-14
Growth Rate:
-87.50%
Total runs:
15
Run Growth:
-3
Growth Rate:
-20.00%
Total runs:
11
Run Growth:
-1
Growth Rate:
-9.09%