Total runs:
29.7K
Run Growth:
5.0K
Growth Rate:
16.74%
Sponsored by
Agentkit
BSC-LT / ALIA-40b-instruct-2605-GGUF
Model's Last Updated: May 11 2026
text-generation
Introduction of ALIA-40b-instruct-2605-GGUF
Model Details of ALIA-40b-instruct-2605-GGUF
Work In Progress New versions will be available during the coming weeks/months.
Sampling Parameters: For optimal performance, we recommend using temperatures close to zero (0 - 0.2). Additionally, we advise against using any type of repetition penalty, as from our experience, it negatively impacts instructed model's responses .
ALIA-40b-instruct-GGUF
- Model creator: BSC-LT .
- Original model: ALIA-40b-instruct-2605
Description
This repo contains GGUF format model files for ALIA-40b-instruct-2605 .
Quantization
Model weights were exported to
GGUF in FP16
first, then quantized with
llama.cpp
’s
llama-quantize
into the target preset (e.g.,
Q8
). The same conversion + quantization pipeline was executed through
quantool
from a single YAML config (e.g.,
method: gguf
,
quant_level
, and method-specific
quantization_config
) and run via
quantool config.yaml
.
About GGUF
GGUF is the model file format introduced by the llama.cpp team on August 21st, 2023 , replacing the older GGML format (now deprecated). It brings significant improvements such as enhanced tokenization, proper handling of special tokens, embedded metadata (e.g., architecture, quantization type, tokenizer), and an extensible design for future compatibility.
How to use
Deployment as service and remote use (Messages API)
In our experience,
vllm
works well for deploying the full unquantized version of the model, whereas
llama.cpp
is appropriate for the quantized (GGUF) version.
We strongly discourage using ollama as we have encountered compatibility issues that may seriously degrade the model's performance.
The easiest and most reliable way to have a working deployment of ALIA-40b-instruct is through the "Deploy / HF Inference Endpoints" option directly on the Hugging Face model page. This automatically creates a functioning endpoint, using vllm or llama.cpp according to the model variant, with an appropriately dimensioned GPU. While there are additional settings available for the endpoint we found the standard configuration proposed by Hugging Face to be a reasonable starting point.
Once the endpoint is running, the model can be easily called using OpenAI's "Messages API" (the de facto standard API for LLM use). By using this API the chat template is applied automatically by the service, requiring no explicit configuration on the client side. The endpoint's configuration page on Hugging Face also provides a "Playground" for testing and API examples, as well as a simple chat interface.
Example usage:
# pip install openai
from openai import OpenAI
client = OpenAI(
base_url = YOUR_ENDPOINT_URL,
api_key = "$HF_TOKEN"
)
chat_completion = client.chat.completions.create(
model = "BSC-LT/ALIA-40b-instruct-2605-GGUF",
messages = [
{
"role": "user",
"content": "What is deep learning?"
}
],
stream = True,
max_tokens = 1000,
temperature=0.1
)
print(chat_completion.choices[0].message.content)
The model can also be deployed locally or on any server infrastructure with sufficient GPUs, using vllm or llama.cpp. We recommend an initial deployment on Hugging Face as a point of reference and comparison to make sure the model is behaving as expected in the desired deployment setup.
To check that your endpoint is working correctly, you can try to replicate the examples contained in this Colab Notebook
Evaluation
WIP
Additional information
Author
The Language Modeling team from AI Institute at Barcelona Supercomputing Center.
Contact
For further information, please send an email to [email protected] .
Copyright
Copyright(c) 2026 by The Language Modeling team from AI Institute at Barcelona Supercomputing Center.
Funding
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Modelos del Lenguaje.
This work has been promoted and supported by the Government of Catalonia through the Aina Project.
Acknowledgements
This project has benefited from the contributions of numerous teams and institutions, mainly through data contributions, knowledge transfer or technical support.
We are especially grateful to our ILENIA project partners: CENID, HiTZ and CiTIUS for their participation. We also extend our genuine gratitude to the Spanish Senate and Congress, Fundación Dialnet, and the ‘Instituto Universitario de Sistemas Inteligentes y Aplicaciones Numéricas en Ingeniería (SIANI)’ of the University of Las Palmas de Gran Canaria. Many other institutions have been involved in the project. Our thanks to Òmnium Cultural, Parlament de Catalunya, Institut d'Estudis Aranesos, Racó Català, Vilaweb, ACN, Nació Digital, El món and Aquí Berguedà. We thank the Welsh government, DFKI, Occiglot project, especially Malte Ostendorff, and The Common Crawl Foundation, especially Pedro Ortiz, for their collaboration.
We would also like to give special thanks to the NVIDIA team, with whom we have met regularly, especially to: Marcelo Sanchez, Ignacio Sarasua, Adam Henryk Grzywaczewski, Oleg Sudakov, Sergio Perez, Miguel Martinez, Felipe Soares and Meriem Bendris. Their constant support has been especially appreciated throughout the entire process.
Their valuable efforts have been instrumental in the development of this work.
Disclaimer
Be aware that the model may contain biases or other unintended distortions. When third parties deploy systems or provide services based on this model, or use the model themselves, they bear the responsibility for mitigating any associated risks and ensuring compliance with applicable regulations, including those governing the use of Artificial Intelligence.
The Barcelona Supercomputing Center, as the owner and creator of the model, shall not be held liable for any outcomes resulting from third-party use.
Citation
@misc{gonzalezagirre2025salamandratechnicalreport,
title={Salamandra Technical Report},
author={Aitor Gonzalez-Agirre and Marc Pàmies and Joan Llop and Irene Baucells and Severino Da Dalt and Daniel Tamayo and José Javier Saiz and Ferran Espuña and Jaume Prats and Javier Aula-Blasco and Mario Mina and Adrián Rubio and Alexander Shvets and Anna Sallés and Iñaki Lacunza and Iñigo Pikabea and Jorge Palomar and Júlia Falcão and Lucía Tormo and Luis Vasquez-Reina and Montserrat Marimon and Valle Ruíz-Fernández and Marta Villegas},
year={2025},
eprint={2502.08489},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.08489},
}
License
Model Index
Runs of BSC-LT ALIA-40b-instruct-2605-GGUF on huggingface.co
19
Total runs
7
24-hour runs
9
3-day runs
8
7-day runs
0
30-day runs
More Information About ALIA-40b-instruct-2605-GGUF huggingface.co Model
Url of ALIA-40b-instruct-2605-GGUF
ALIA-40b-instruct-2605-GGUF huggingface.co Url
Provider of ALIA-40b-instruct-2605-GGUF huggingface.co
Other API from BSC-LT
Updated:October 22 2025
Total runs:
2.3K
Run Growth:
-865
Growth Rate:
-37.32%
Total runs:
1.7K
Run Growth:
-697
Growth Rate:
-40.83%
Total runs:
1.6K
Run Growth:
-3.6K
Growth Rate:
-229.90%
Total runs:
1.4K
Run Growth:
-2.3K
Growth Rate:
-287.84%
Total runs:
1.3K
Run Growth:
-157
Growth Rate:
-12.37%
Total runs:
814
Run Growth:
-466
Growth Rate:
-58.03%
Total runs:
692
Run Growth:
555
Growth Rate:
81.50%
Total runs:
567
Run Growth:
300
Growth Rate:
52.91%
Total runs:
503
Run Growth:
493
Growth Rate:
98.01%
Total runs:
472
Run Growth:
68
Growth Rate:
14.41%
Total runs:
453
Run Growth:
-670
Growth Rate:
-143.47%
Total runs:
354
Run Growth:
154
Growth Rate:
57.04%
Total runs:
336
Run Growth:
-726
Growth Rate:
-210.43%
Total runs:
245
Run Growth:
101
Growth Rate:
41.22%
Total runs:
237
Run Growth:
27
Growth Rate:
11.39%
Total runs:
234
Run Growth:
229
Growth Rate:
97.86%
Total runs:
206
Run Growth:
-2
Growth Rate:
-0.97%
Total runs:
203
Run Growth:
1
Growth Rate:
0.54%
Total runs:
180
Run Growth:
5
Growth Rate:
2.67%
Total runs:
174
Run Growth:
33
Growth Rate:
18.97%
Total runs:
147
Run Growth:
103
Growth Rate:
70.55%
Total runs:
146
Run Growth:
-131
Growth Rate:
-89.73%
Total runs:
145
Run Growth:
-161
Growth Rate:
-111.03%
Total runs:
142
Run Growth:
-46
Growth Rate:
-32.39%
Total runs:
137
Run Growth:
-666
Growth Rate:
-486.13%
Total runs:
136
Run Growth:
91
Growth Rate:
66.91%
Total runs:
131
Run Growth:
95
Growth Rate:
72.52%
Total runs:
129
Run Growth:
-14
Growth Rate:
-10.85%
Total runs:
127
Run Growth:
-58
Growth Rate:
-45.67%
Total runs:
124
Run Growth:
83
Growth Rate:
66.94%
Total runs:
120
Run Growth:
-85
Growth Rate:
-70.83%
Total runs:
119
Run Growth:
94
Growth Rate:
83.19%
Total runs:
117
Run Growth:
65
Growth Rate:
54.62%
Total runs:
113
Run Growth:
-169
Growth Rate:
-156.48%
Total runs:
107
Run Growth:
37
Growth Rate:
35.24%
Total runs:
107
Run Growth:
75
Growth Rate:
75.76%
Total runs:
106
Run Growth:
62
Growth Rate:
58.49%
Total runs:
105
Run Growth:
69
Growth Rate:
65.71%
Total runs:
105
Run Growth:
57
Growth Rate:
54.29%
Total runs:
102
Run Growth:
-23
Growth Rate:
-22.55%
Total runs:
97
Run Growth:
57
Growth Rate:
58.76%
Total runs:
97
Run Growth:
54
Growth Rate:
55.67%
Total runs:
97
Run Growth:
78
Growth Rate:
83.87%
Total runs:
95
Run Growth:
50
Growth Rate:
53.19%
Total runs:
92
Run Growth:
91
Growth Rate:
98.91%
Total runs:
91
Run Growth:
32
Growth Rate:
36.78%
Total runs:
86
Run Growth:
67
Growth Rate:
80.72%
Total runs:
86
Run Growth:
73
Growth Rate:
90.12%
Total runs:
84
Run Growth:
71
Growth Rate:
87.65%
Total runs:
79
Run Growth:
69
Growth Rate:
88.46%
Total runs:
73
Run Growth:
61
Growth Rate:
83.56%
Total runs:
73
Run Growth:
58
Growth Rate:
81.69%
Total runs:
51
Run Growth:
28
Growth Rate:
54.90%
Total runs:
49
Run Growth:
-25
Growth Rate:
-51.02%
Total runs:
47
Run Growth:
-64
Growth Rate:
-145.45%
Total runs:
47
Run Growth:
-8
Growth Rate:
-17.39%
Total runs:
45
Run Growth:
4
Growth Rate:
12.12%
Total runs:
43
Run Growth:
0
Growth Rate:
0.00%
Total runs:
40
Run Growth:
-26
Growth Rate:
-65.00%
Total runs:
39
Run Growth:
-6
Growth Rate:
-15.38%
Total runs:
36
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
-21
Growth Rate:
-67.74%
Total runs:
28
Run Growth:
11
Growth Rate:
45.83%
Total runs:
28
Run Growth:
-14
Growth Rate:
-50.00%
Total runs:
27
Run Growth:
-16
Growth Rate:
-59.26%
Total runs:
26
Run Growth:
-12
Growth Rate:
-46.15%
Total runs:
26
Run Growth:
-28
Growth Rate:
-107.69%
Total runs:
26
Run Growth:
-16
Growth Rate:
-61.54%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
21
Run Growth:
9
Growth Rate:
42.86%
Total runs:
19
Run Growth:
-201
Growth Rate:
-1057.89%
Total runs:
18
Run Growth:
2
Growth Rate:
10.00%
Total runs:
16
Run Growth:
-14
Growth Rate:
-87.50%
Total runs:
15
Run Growth:
-3
Growth Rate:
-20.00%
Total runs:
11
Run Growth:
-1
Growth Rate:
-9.09%