Total runs:
10.4K
Run Growth:
805
Growth Rate:
7.71%
Sponsored by
APIMart
speakleash / Bielik-11B-v2.5-Instruct-FP8-Dynamic
Model's Last Updated: June 06 2025
text-generation
Introduction of Bielik-11B-v2.5-Instruct-FP8-Dynamic
Model Details of Bielik-11B-v2.5-Instruct-FP8-Dynamic
Bielik-11B-v2.5-Instruct-FP8-Dynamic
This model was obtained by quantizing the weights and activations of Bielik-11B-v2.5-Instruct to FP8 data type, ready for inference with vLLM >= 0.5.0 or SGLang. AutoFP8 is used for quantization. This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformers blocks are quantized. Symmetric per-tensor quantization is applied, in which a single linear scaling maps the FP8 representations of the quantized weights and activations.
FP8 compuation is supported on Nvidia GPUs with compute capability > 8.9 (Ada Lovelace, Hopper).
DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!
Use with vLLM
This model can be deployed efficiently using the vLLM backend, as shown in the example below.
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
model_id = "speakleash/Bielik-11B-v2.5-Instruct-FP8-Dynamic"
sampling_params = SamplingParams(temperature=0.2, top_p=0.95, max_tokens=4096)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
]
prompts = tokenizer.apply_chat_template(messages, tokenize=False)
llm = LLM(model=model_id, max_model_len=4096)
outputs = llm.generate(prompts, sampling_params)
generated_text = outputs[0].outputs[0].text
print(generated_text)
vLLM aslo supports OpenAI-compatible serving. See the documentation for more details.
Use with SGLang Runtime
Launch a server of SGLang Runtime:
python -m sglang.launch_server --model-path speakleash/Bielik-11B-v2.5-Instruct-FP8-Dynamic --port 30000
Then you can send http request or use OpenAI Compatible API.
import openai
client = openai.Client(
base_url="http://127.0.0.1:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="default",
messages=[
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
],
temperature=0,
max_tokens=4096,
)
print(response)
Model description:
- Developed by: SpeakLeash & ACK Cyfronet AGH
- Language: Polish
- Model type: causal decoder-only
- Quant from: Bielik-11B-v2.5-Instruct
- Finetuned from: Bielik-11B-v2
- License: Apache 2.0 and Terms of Use
Responsible for model quantization
- Remigiusz Kinas SpeakLeash - team leadership, conceptualizing, calibration data preparation, process creation and quantized model delivery.
Contact Us
If you have any questions or suggestions, please use the discussion tab. If you want to contact us directly, join our Discord SpeakLeash .
Runs of speakleash Bielik-11B-v2.5-Instruct-FP8-Dynamic on huggingface.co
80
Total runs
-2
24-hour runs
0
3-day runs
7
7-day runs
47
30-day runs
More Information About Bielik-11B-v2.5-Instruct-FP8-Dynamic huggingface.co Model
Url of Bielik-11B-v2.5-Instruct-FP8-Dynamic
Bielik-11B-v2.5-Instruct-FP8-Dynamic huggingface.co Url
Provider of Bielik-11B-v2.5-Instruct-FP8-Dynamic huggingface.co
Other API from speakleash
Updated:January 29 2026
Total runs:
8.5K
Run Growth:
3.9K
Growth Rate:
44.89%
Total runs:
5.6K
Run Growth:
2.0K
Growth Rate:
36.23%
Total runs:
4.7K
Run Growth:
-213.1K
Growth Rate:
-4211.66%
Total runs:
4.2K
Run Growth:
-5.5K
Growth Rate:
-132.53%
Total runs:
3.2K
Run Growth:
1.3K
Growth Rate:
41.56%
Total runs:
3.2K
Run Growth:
-28
Growth Rate:
-0.88%
Total runs:
3.2K
Run Growth:
-27
Growth Rate:
-0.86%
Total runs:
2.6K
Run Growth:
-6.4K
Growth Rate:
-245.67%
Total runs:
2.4K
Run Growth:
-2.2K
Growth Rate:
-91.71%
Total runs:
2.0K
Run Growth:
242
Growth Rate:
12.28%
Total runs:
1.6K
Run Growth:
553
Growth Rate:
33.47%
Total runs:
1.5K
Run Growth:
97
Growth Rate:
6.28%
Total runs:
1.5K
Run Growth:
527
Growth Rate:
35.46%
Total runs:
1.4K
Run Growth:
473
Growth Rate:
35.04%
Total runs:
1.2K
Run Growth:
-600
Growth Rate:
-50.34%
Total runs:
1.1K
Run Growth:
-49
Growth Rate:
-4.58%
Total runs:
990
Run Growth:
362
Growth Rate:
37.17%
Total runs:
967
Run Growth:
-227
Growth Rate:
-23.31%
Total runs:
861
Run Growth:
416
Growth Rate:
48.60%
Total runs:
860
Run Growth:
258
Growth Rate:
30.07%
Total runs:
832
Run Growth:
-74
Growth Rate:
-8.96%
Total runs:
651
Run Growth:
-21
Growth Rate:
-3.23%
Total runs:
614
Run Growth:
36
Growth Rate:
5.95%
Total runs:
584
Run Growth:
102
Growth Rate:
17.38%
Total runs:
580
Run Growth:
403
Growth Rate:
67.39%
Total runs:
542
Run Growth:
-230
Growth Rate:
-42.44%
Total runs:
538
Run Growth:
59
Growth Rate:
11.48%
Total runs:
503
Run Growth:
-178
Growth Rate:
-35.39%
Total runs:
496
Run Growth:
-87
Growth Rate:
-17.54%
Total runs:
486
Run Growth:
-73
Growth Rate:
-15.02%
Total runs:
483
Run Growth:
18
Growth Rate:
3.73%
Total runs:
430
Run Growth:
296
Growth Rate:
69.00%
Total runs:
380
Run Growth:
-2.5K
Growth Rate:
-652.11%
Total runs:
357
Run Growth:
327
Growth Rate:
92.11%
Total runs:
310
Run Growth:
-185
Growth Rate:
-74.90%
Total runs:
304
Run Growth:
75
Growth Rate:
24.67%
Total runs:
297
Run Growth:
-78
Growth Rate:
-29.21%
Total runs:
268
Run Growth:
3
Growth Rate:
1.13%
Total runs:
249
Run Growth:
-171
Growth Rate:
-68.67%
Total runs:
248
Run Growth:
198
Growth Rate:
79.84%
Total runs:
241
Run Growth:
-7
Growth Rate:
-2.99%
Total runs:
240
Run Growth:
-22
Growth Rate:
-9.17%
Total runs:
234
Run Growth:
71
Growth Rate:
30.34%
Total runs:
210
Run Growth:
138
Growth Rate:
63.89%
Total runs:
189
Run Growth:
-11
Growth Rate:
-5.64%
Total runs:
181
Run Growth:
26
Growth Rate:
14.36%
Total runs:
170
Run Growth:
43
Growth Rate:
25.90%
Total runs:
158
Run Growth:
10
Growth Rate:
6.45%
Total runs:
156
Run Growth:
-149
Growth Rate:
-96.13%
Total runs:
149
Run Growth:
-180
Growth Rate:
-112.50%
Total runs:
139
Run Growth:
74
Growth Rate:
67.27%
Total runs:
137
Run Growth:
14
Growth Rate:
10.22%
Total runs:
127
Run Growth:
-470
Growth Rate:
-370.08%
Total runs:
125
Run Growth:
-36
Growth Rate:
-28.80%
Total runs:
115
Run Growth:
-37
Growth Rate:
-32.17%
Total runs:
96
Run Growth:
61
Growth Rate:
63.54%
Total runs:
95
Run Growth:
21
Growth Rate:
22.11%
Total runs:
91
Run Growth:
28
Growth Rate:
30.77%
Total runs:
88
Run Growth:
40
Growth Rate:
45.45%
Total runs:
85
Run Growth:
50
Growth Rate:
58.82%
Total runs:
84
Run Growth:
-90
Growth Rate:
-107.14%
Total runs:
82
Run Growth:
24
Growth Rate:
29.27%
Total runs:
78
Run Growth:
-37
Growth Rate:
-47.44%
Total runs:
78
Run Growth:
45
Growth Rate:
57.69%
Total runs:
76
Run Growth:
-1
Growth Rate:
-1.32%
Total runs:
75
Run Growth:
13
Growth Rate:
17.33%
Total runs:
74
Run Growth:
-12
Growth Rate:
-17.65%
Total runs:
74
Run Growth:
-7
Growth Rate:
-9.72%
Total runs:
70
Run Growth:
-1
Growth Rate:
-1.43%
Total runs:
68
Run Growth:
35
Growth Rate:
50.72%
Total runs:
59
Run Growth:
19
Growth Rate:
32.20%
Total runs:
45
Run Growth:
37
Growth Rate:
86.05%
Total runs:
36
Run Growth:
18
Growth Rate:
50.00%
Total runs:
30
Run Growth:
-1
Growth Rate:
-3.33%
Total runs:
28
Run Growth:
2
Growth Rate:
7.14%
Total runs:
19
Run Growth:
-7
Growth Rate:
-36.84%
Total runs:
17
Run Growth:
-3
Growth Rate:
-17.65%
Total runs:
11
Run Growth:
1
Growth Rate:
9.09%
Total runs:
11
Run Growth:
-22
Growth Rate:
-200.00%
Total runs:
9
Run Growth:
-2
Growth Rate:
-22.22%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
0
Run Growth:
-2
Growth Rate:
0.00%