Total runs:
232.9K
Run Growth:
-4.6K
Growth Rate:
-1.98%
Sponsored by
Agentkit
speakleash / Bielik-11B-v2.2-Instruct-FP8
Model's Last Updated: June 06 2025
text-generation
Introduction of Bielik-11B-v2.2-Instruct-FP8
Model Details of Bielik-11B-v2.2-Instruct-FP8
Bielik-11B-v2.2-Instruct-FP8
This model was obtained by quantizing the weights and activations of Bielik-11B-v.2.2-Instruct to FP8 data type, ready for inference with vLLM >= 0.5.0 or SGLang. AutoFP8 is used for quantization. This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformers blocks are quantized. Symmetric per-tensor quantization is applied, in which a single linear scaling maps the FP8 representations of the quantized weights and activations.
FP8 compuation is supported on Nvidia GPUs with compute capability > 8.9 (Ada Lovelace, Hopper).
DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!
Use with vLLM
This model can be deployed efficiently using the vLLM backend, as shown in the example below.
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
model_id = "speakleash/Bielik-11B-v2.2-Instruct-FP8"
sampling_params = SamplingParams(temperature=0.2, top_p=0.95, max_tokens=4096)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
]
prompts = tokenizer.apply_chat_template(messages, tokenize=False)
llm = LLM(model=model_id, max_model_len=4096)
outputs = llm.generate(prompts, sampling_params)
generated_text = outputs[0].outputs[0].text
print(generated_text)
vLLM aslo supports OpenAI-compatible serving. See the documentation for more details.
Use with SGLang Runtime
Launch a server of SGLang Runtime:
python -m sglang.launch_server --model-path speakleash/Bielik-11B-v2.2-Instruct-FP8 --port 30000
Then you can send http request or use OpenAI Compatible API.
import openai
client = openai.Client(
base_url="http://127.0.0.1:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="default",
messages=[
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
],
temperature=0,
max_tokens=4096,
)
print(response)
Model description:
- Developed by: SpeakLeash & ACK Cyfronet AGH
- Language: Polish
- Model type: causal decoder-only
- Quant from: Bielik-11B-v2.2-Instruct
- Finetuned from: Bielik-11B-v2
- License: Apache 2.0 and Terms of Use
Responsible for model quantization
- Remigiusz Kinas SpeakLeash - team leadership, conceptualizing, calibration data preparation, process creation and quantized model delivery.
Contact Us
If you have any questions or suggestions, please use the discussion tab. If you want to contact us directly, join our Discord SpeakLeash .
Runs of speakleash Bielik-11B-v2.2-Instruct-FP8 on huggingface.co
45
Total runs
0
24-hour runs
1
3-day runs
9
7-day runs
4
30-day runs
More Information About Bielik-11B-v2.2-Instruct-FP8 huggingface.co Model
Url of Bielik-11B-v2.2-Instruct-FP8
Bielik-11B-v2.2-Instruct-FP8 huggingface.co Url
Provider of Bielik-11B-v2.2-Instruct-FP8 huggingface.co
Other API from speakleash
Updated:December 31 2025
Total runs:
9.6K
Run Growth:
-47.1K
Growth Rate:
-490.11%
Total runs:
8.9K
Run Growth:
5.8K
Growth Rate:
65.16%
Total runs:
8.1K
Run Growth:
-281.4K
Growth Rate:
-3482.67%
Total runs:
4.7K
Run Growth:
-3.5K
Growth Rate:
-74.61%
Total runs:
4.7K
Run Growth:
-1.4K
Growth Rate:
-29.58%
Total runs:
3.6K
Run Growth:
904
Growth Rate:
25.44%
Total runs:
3.1K
Run Growth:
-225
Growth Rate:
-7.15%
Total runs:
3.1K
Run Growth:
973
Growth Rate:
31.81%
Total runs:
1.9K
Run Growth:
-2.0K
Growth Rate:
-108.10%
Total runs:
1.9K
Run Growth:
-1.1K
Growth Rate:
-57.19%
Total runs:
1.4K
Run Growth:
607
Growth Rate:
43.67%
Total runs:
1.3K
Run Growth:
-55
Growth Rate:
-4.12%
Total runs:
1.2K
Run Growth:
300
Growth Rate:
24.55%
Total runs:
1.0K
Run Growth:
632
Growth Rate:
61.60%
Total runs:
1.0K
Run Growth:
160
Growth Rate:
15.92%
Total runs:
957
Run Growth:
-542
Growth Rate:
-56.64%
Total runs:
914
Run Growth:
405
Growth Rate:
44.31%
Total runs:
803
Run Growth:
-12
Growth Rate:
-1.49%
Total runs:
722
Run Growth:
357
Growth Rate:
49.45%
Total runs:
698
Run Growth:
73
Growth Rate:
10.46%
Total runs:
645
Run Growth:
162
Growth Rate:
25.12%
Total runs:
631
Run Growth:
188
Growth Rate:
29.79%
Total runs:
594
Run Growth:
510
Growth Rate:
85.86%
Total runs:
564
Run Growth:
297
Growth Rate:
52.66%
Total runs:
553
Run Growth:
171
Growth Rate:
30.92%
Total runs:
544
Run Growth:
80
Growth Rate:
14.71%
Total runs:
530
Run Growth:
221
Growth Rate:
41.70%
Total runs:
493
Run Growth:
-318
Growth Rate:
-64.50%
Total runs:
464
Run Growth:
53
Growth Rate:
11.42%
Total runs:
454
Run Growth:
208
Growth Rate:
45.81%
Total runs:
444
Run Growth:
-1.4K
Growth Rate:
-311.04%
Total runs:
425
Run Growth:
235
Growth Rate:
55.29%
Total runs:
415
Run Growth:
-149
Growth Rate:
-35.90%
Total runs:
349
Run Growth:
-338
Growth Rate:
-96.85%
Total runs:
342
Run Growth:
-455
Growth Rate:
-133.04%
Total runs:
321
Run Growth:
259
Growth Rate:
80.69%
Total runs:
319
Run Growth:
-68
Growth Rate:
-21.32%
Total runs:
279
Run Growth:
44
Growth Rate:
15.77%
Total runs:
272
Run Growth:
90
Growth Rate:
33.09%
Total runs:
264
Run Growth:
11
Growth Rate:
4.17%
Total runs:
249
Run Growth:
-25
Growth Rate:
-10.04%
Total runs:
231
Run Growth:
-20
Growth Rate:
-8.66%
Total runs:
204
Run Growth:
76
Growth Rate:
37.25%
Total runs:
162
Run Growth:
64
Growth Rate:
39.51%
Total runs:
156
Run Growth:
43
Growth Rate:
27.56%
Total runs:
156
Run Growth:
16
Growth Rate:
10.26%
Total runs:
146
Run Growth:
-13
Growth Rate:
-8.90%
Total runs:
142
Run Growth:
-340
Growth Rate:
-239.44%
Total runs:
139
Run Growth:
37
Growth Rate:
26.62%
Total runs:
128
Run Growth:
-209
Growth Rate:
-163.28%
Total runs:
126
Run Growth:
-6
Growth Rate:
-4.76%
Total runs:
119
Run Growth:
15
Growth Rate:
12.61%
Total runs:
82
Run Growth:
26
Growth Rate:
31.71%
Total runs:
79
Run Growth:
-276
Growth Rate:
-349.37%
Total runs:
77
Run Growth:
0
Growth Rate:
0.00%
Total runs:
76
Run Growth:
39
Growth Rate:
51.32%
Total runs:
76
Run Growth:
22
Growth Rate:
28.95%
Total runs:
70
Run Growth:
15
Growth Rate:
21.43%
Total runs:
70
Run Growth:
16
Growth Rate:
22.86%
Total runs:
62
Run Growth:
10
Growth Rate:
16.13%
Total runs:
57
Run Growth:
2
Growth Rate:
3.51%
Total runs:
56
Run Growth:
19
Growth Rate:
33.93%
Total runs:
44
Run Growth:
15
Growth Rate:
34.09%
Total runs:
40
Run Growth:
14
Growth Rate:
35.00%
Total runs:
37
Run Growth:
2
Growth Rate:
5.41%
Total runs:
37
Run Growth:
-39
Growth Rate:
-105.41%
Total runs:
35
Run Growth:
1
Growth Rate:
2.86%
Total runs:
33
Run Growth:
11
Growth Rate:
33.33%
Total runs:
32
Run Growth:
17
Growth Rate:
53.13%
Total runs:
32
Run Growth:
2
Growth Rate:
6.25%
Total runs:
31
Run Growth:
20
Growth Rate:
64.52%
Total runs:
31
Run Growth:
-75
Growth Rate:
-241.94%
Total runs:
28
Run Growth:
-104
Growth Rate:
-371.43%
Total runs:
24
Run Growth:
15
Growth Rate:
62.50%
Total runs:
24
Run Growth:
10
Growth Rate:
41.67%
Total runs:
18
Run Growth:
-74
Growth Rate:
-411.11%
Total runs:
18
Run Growth:
8
Growth Rate:
44.44%
Total runs:
9
Run Growth:
4
Growth Rate:
44.44%
Total runs:
7
Run Growth:
1
Growth Rate:
14.29%
Total runs:
6
Run Growth:
1
Growth Rate:
16.67%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
0
Run Growth:
-2
Growth Rate:
0.00%