Total runs:
233.5K
Run Growth:
8.9K
Growth Rate:
3.81%
Sponsored by
APIMart
speakleash / Bielik-4.5B-v3.0-Instruct-FP8-Dynamic
Model's Last Updated: January 29 2026
text-generation
Introduction of Bielik-4.5B-v3.0-Instruct-FP8-Dynamic
Model Details of Bielik-4.5B-v3.0-Instruct-FP8-Dynamic
Bielik-4.5B-v3.0-Instruct-FP8-Dynamic
This model was obtained by quantizing the weights and activations of Bielik-4.5B-v3.0-Instruct to FP8 data type, ready for inference with vLLM >= 0.5.0 or SGLang. AutoFP8 is used for quantization. This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformers blocks are quantized. Symmetric per-tensor quantization is applied, in which a single linear scaling maps the FP8 representations of the quantized weights and activations.
📚 Technical report: https://arxiv.org/abs/2505.02550
FP8 compuation is supported on Nvidia GPUs with compute capability > 8.9 (Ada Lovelace, Hopper).
DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!
Use with vLLM
This model can be deployed efficiently using the vLLM backend, as shown in the example below.
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
model_id = "speakleash/Bielik-4.5B-v3.0-Instruct-FP8-Dynamic"
sampling_params = SamplingParams(temperature=0.2, top_p=0.95, max_tokens=4096)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
]
prompts = tokenizer.apply_chat_template(messages, tokenize=False)
llm = LLM(model=model_id, max_model_len=4096)
outputs = llm.generate(prompts, sampling_params)
generated_text = outputs[0].outputs[0].text
print(generated_text)
vLLM aslo supports OpenAI-compatible serving. See the documentation for more details.
Use with SGLang Runtime
Launch a server of SGLang Runtime:
python -m sglang.launch_server --model-path speakleash/Bielik-4.5B-v3.0-Instruct-FP8-Dynamic --port 30000
Then you can send http request or use OpenAI Compatible API.
import openai
client = openai.Client(
base_url="http://127.0.0.1:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="default",
messages=[
{"role": "system", "content": "Jesteś pomocnym asystentem Bielik."},
{"role": "user", "content": "Kim był Mikołaj Kopernik i z czego zasłynął?"},
],
temperature=0,
max_tokens=4096,
)
print(response)
Model description:
- Developed by: SpeakLeash & ACK Cyfronet AGH
- Language: Polish
- Model type: causal decoder-only
- Quant from: Bielik-4.5B-v3.0-Instruct
- Finetuned from: Bielik-4.5B-v3
- License: Apache 2.0 and Terms of Use
Responsible for model quantization
- Remigiusz Kinas SpeakLeash - team leadership, conceptualizing, calibration data preparation, process creation and quantized model delivery.
Contact Us
If you have any questions or suggestions, please use the discussion tab. If you want to contact us directly, join our Discord SpeakLeash .
Runs of speakleash Bielik-4.5B-v3.0-Instruct-FP8-Dynamic on huggingface.co
906
Total runs
152
24-hour runs
138
3-day runs
186
7-day runs
506
30-day runs
More Information About Bielik-4.5B-v3.0-Instruct-FP8-Dynamic huggingface.co Model
Url of Bielik-4.5B-v3.0-Instruct-FP8-Dynamic
Bielik-4.5B-v3.0-Instruct-FP8-Dynamic huggingface.co Url
Provider of Bielik-4.5B-v3.0-Instruct-FP8-Dynamic huggingface.co
Other API from speakleash
Updated:December 31 2025
Total runs:
9.6K
Run Growth:
-87.0K
Growth Rate:
-909.65%
Total runs:
8.9K
Run Growth:
6.3K
Growth Rate:
70.02%
Total runs:
5.1K
Run Growth:
-2.1K
Growth Rate:
-38.64%
Total runs:
4.8K
Run Growth:
-349.1K
Growth Rate:
-7047.81%
Total runs:
4.7K
Run Growth:
-1.3K
Growth Rate:
-27.93%
Total runs:
3.2K
Run Growth:
1.6K
Growth Rate:
50.05%
Total runs:
3.1K
Run Growth:
105
Growth Rate:
3.34%
Total runs:
2.0K
Run Growth:
-1.4K
Growth Rate:
-71.29%
Total runs:
1.9K
Run Growth:
-789
Growth Rate:
-42.37%
Total runs:
1.4K
Run Growth:
744
Growth Rate:
53.60%
Total runs:
1.3K
Run Growth:
2
Growth Rate:
0.15%
Total runs:
1.2K
Run Growth:
134
Growth Rate:
11.16%
Total runs:
1.1K
Run Growth:
429
Growth Rate:
40.13%
Total runs:
985
Run Growth:
568
Growth Rate:
58.80%
Total runs:
952
Run Growth:
-428
Growth Rate:
-44.86%
Total runs:
819
Run Growth:
-80
Growth Rate:
-10.23%
Total runs:
703
Run Growth:
346
Growth Rate:
51.18%
Total runs:
691
Run Growth:
-314
Growth Rate:
-46.87%
Total runs:
668
Run Growth:
332
Growth Rate:
50.30%
Total runs:
632
Run Growth:
207
Growth Rate:
34.21%
Total runs:
587
Run Growth:
441
Growth Rate:
76.70%
Total runs:
561
Run Growth:
147
Growth Rate:
26.25%
Total runs:
559
Run Growth:
175
Growth Rate:
41.97%
Total runs:
557
Run Growth:
296
Growth Rate:
55.02%
Total runs:
541
Run Growth:
150
Growth Rate:
28.14%
Total runs:
471
Run Growth:
-297
Growth Rate:
-62.92%
Total runs:
464
Run Growth:
42
Growth Rate:
9.07%
Total runs:
454
Run Growth:
-1.7K
Growth Rate:
-366.45%
Total runs:
450
Run Growth:
-2.6K
Growth Rate:
-542.56%
Total runs:
437
Run Growth:
314
Growth Rate:
72.35%
Total runs:
436
Run Growth:
186
Growth Rate:
42.08%
Total runs:
404
Run Growth:
-111
Growth Rate:
-27.14%
Total runs:
351
Run Growth:
-257
Growth Rate:
-73.43%
Total runs:
342
Run Growth:
-610
Growth Rate:
-184.85%
Total runs:
335
Run Growth:
-252
Growth Rate:
-74.56%
Total runs:
322
Run Growth:
261
Growth Rate:
80.56%
Total runs:
287
Run Growth:
98
Growth Rate:
35.77%
Total runs:
267
Run Growth:
78
Growth Rate:
28.06%
Total runs:
248
Run Growth:
58
Growth Rate:
23.11%
Total runs:
240
Run Growth:
-31
Growth Rate:
-12.50%
Total runs:
234
Run Growth:
-72
Growth Rate:
-30.13%
Total runs:
195
Run Growth:
65
Growth Rate:
34.39%
Total runs:
159
Run Growth:
62
Growth Rate:
38.75%
Total runs:
159
Run Growth:
70
Growth Rate:
44.87%
Total runs:
157
Run Growth:
24
Growth Rate:
16.00%
Total runs:
149
Run Growth:
6
Growth Rate:
4.03%
Total runs:
146
Run Growth:
127
Growth Rate:
65.13%
Total runs:
139
Run Growth:
-399
Growth Rate:
-282.98%
Total runs:
134
Run Growth:
40
Growth Rate:
30.77%
Total runs:
128
Run Growth:
-223
Growth Rate:
-176.98%
Total runs:
119
Run Growth:
2
Growth Rate:
1.75%
Total runs:
79
Run Growth:
-294
Growth Rate:
-372.15%
Total runs:
77
Run Growth:
52
Growth Rate:
63.41%
Total runs:
72
Run Growth:
25
Growth Rate:
33.78%
Total runs:
72
Run Growth:
22
Growth Rate:
31.43%
Total runs:
71
Run Growth:
20
Growth Rate:
29.41%
Total runs:
70
Run Growth:
-4
Growth Rate:
-5.63%
Total runs:
62
Run Growth:
15
Growth Rate:
26.79%
Total runs:
58
Run Growth:
6
Growth Rate:
10.91%
Total runs:
55
Run Growth:
19
Growth Rate:
35.85%
Total runs:
45
Run Growth:
5
Growth Rate:
11.11%
Total runs:
43
Run Growth:
-14
Growth Rate:
-37.84%
Total runs:
42
Run Growth:
5
Growth Rate:
12.20%
Total runs:
39
Run Growth:
3
Growth Rate:
7.69%
Total runs:
38
Run Growth:
14
Growth Rate:
36.84%
Total runs:
37
Run Growth:
-19
Growth Rate:
-51.35%
Total runs:
35
Run Growth:
0
Growth Rate:
0.00%
Total runs:
33
Run Growth:
6
Growth Rate:
19.35%
Total runs:
33
Run Growth:
-70
Growth Rate:
-218.75%
Total runs:
32
Run Growth:
14
Growth Rate:
43.75%
Total runs:
31
Run Growth:
21
Growth Rate:
67.74%
Total runs:
30
Run Growth:
3
Growth Rate:
9.38%
Total runs:
26
Run Growth:
-83
Growth Rate:
-267.74%
Total runs:
21
Run Growth:
16
Growth Rate:
76.19%
Total runs:
21
Run Growth:
5
Growth Rate:
29.41%
Total runs:
18
Run Growth:
-106
Growth Rate:
-588.89%
Total runs:
18
Run Growth:
11
Growth Rate:
64.71%
Total runs:
9
Run Growth:
3
Growth Rate:
33.33%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
6
Run Growth:
3
Growth Rate:
50.00%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
0
Run Growth:
-2
Growth Rate:
0.00%