Total runs:
1.4K
Run Growth:
102
Growth Rate:
6.80%
Sponsored by
PoYo.AI
AXERA-TECH / SmolLM2-360M-Instruct
Model's Last Updated: October 22 2025
Introduction of SmolLM2-360M-Instruct
Model Details of SmolLM2-360M-Instruct
SmolLM2-360M-Instruct
This version of SmolLM2-360M-Instruct has been converted to run on the Axera NPU using w8a16 quantization.
This model has been optimized with the following LoRA:
Compatible with Pulsar2 version: 3.4(Not released yet)
Convert tools links:
For those who are interested in model conversion, you can try to export axmodel through the original repo https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct
Pulsar2 Link, How to Convert LLM from Huggingface to axmodel
Support Platform
-
AX650
- AX650N DEMO Board
- M4N-Dock(爱芯派Pro)
- M.2 Accelerator card
- AX630C
| Chips | w8a16 | w4a16 |
|---|---|---|
| AX650 | 39 tokens/sec | todo |
| AX630C | 14 tokens/sec | todo |
How to use
Download all files from this repository to the device
root@ax650:/mnt/qtang/llm-test/smollm2-360m# tree -L 1
.
|-- main_axcl_aarch64
|-- main_axcl_x86
|-- main_prefill
|-- post_config.json
|-- run_smollm2_360m_ax630c.sh
|-- run_smollm2_360m_ax650.sh
|-- run_smollm2_360m_axcl_aarch64.sh
|-- run_smollm2_360m_axcl_x86.sh
|-- smollm2-360m-ax630c
|-- smollm2-360m-ax650
|-- smollm2_tokenizer
`-- smollm2_tokenizer.py
Install transformer
pip install transformers==4.41.1
Start the Tokenizer service
root@ax650:/mnt/qtang/llm-test/smollm2-360m$ python smollm2_tokenizer.py --port 12345
1 <|im_start|> 2 <|im_end|>
<|im_start|>system
You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>
<|im_start|>user
hello world<|im_end|>
<|im_start|>assistant
[1, 9690, 198, 2683, 359, 253, 5356, 5646, 11173, 3365, 3511, 308, 34519, 28, 7018, 411, 407, 19712, 8182, 2, 198, 1, 4093, 198, 28120, 905, 2, 198, 1, 520, 9531, 198]
http://localhost:12345
Inference with AX650 Host, such as M4N-Dock(爱芯派Pro) or AX650N DEMO Board
Open another terminal and run
run_smollm2_360m_ax650.sh
root@ax650:/mnt/qtang/llm-test/smollm2-360m# ./run_smollm2_360m_ax650.sh
[I][ Init][ 125]: LLM init start
bos_id: 1, eos_id: 2
2% | █ | 1 / 35 [0.00s<0.14s, 250.00 count/s] tokenizer init ok
[I][ Init][ 26]: LLaMaEmbedSelector use mmap
100% | ████████████████████████████████ | 35 / 35 [0.81s<0.81s, 43.37 count/s] init post axmodel ok,remain_cmm(3339 MB)
[I][ Init][ 241]: max_token_len : 1023
[I][ Init][ 246]: kv_cache_size : 320, kv_cache_num: 1023
[I][ Init][ 254]: prefill_token_num : 128
[I][ load_config][ 281]: load config:
{
"enable_repetition_penalty": false,
"enable_temperature": true,
"enable_top_k_sampling": true,
"enable_top_p_sampling": false,
"penalty_window": 20,
"repetition_penalty": 1.2,
"temperature": 0.9,
"top_k": 10,
"top_p": 0.8
}
[I][ Init][ 268]: LLM init ok
Type "q" to exit, Ctrl+c to stop current running
>> who are you?
[I][ Run][ 466]: ttft: 156.63 ms
I'm a chatbot developed by the Artificial Intelligence Research and Development Lab (AI R&D Lab) at Hugging Face Labs,
specifically designed to facilitate and augment human-AI conversations. My role is to provide assistance in understanding
and responding to natural language queries, using advanced language models and AI algorithms to understand context and intent.
[N][ Run][ 605]: hit eos,avg 38.70 token/s
>> q
Inference with M.2 Accelerator card
What is M.2 Accelerator card? , Show this DEMO based on Raspberry PI 5.
(base) axera@raspberrypi:~/samples/smollm2-360m $ ./run_smollm2_360m_axcl_aarch64.sh
build time: Feb 13 2025 15:44:57
[I][ Init][ 111]: LLM init start
bos_id: 1, eos_id: 2
100% | ████████████████████████████████ | 35 / 35 [18.07s<18.07s, 1.94 count/s] init post axmodel okremain_cmm(6621 MB)
[I][ Init][ 226]: max_token_len : 1023
[I][ Init][ 231]: kv_cache_size : 320, kv_cache_num: 1023
[I][ load_config][ 282]: load config:
{
"enable_repetition_penalty": false,
"enable_temperature": true,
"enable_top_k_sampling": true,
"enable_top_p_sampling": false,
"penalty_window": 20,
"repetition_penalty": 1.2,
"temperature": 0.9,
"top_k": 10,
"top_p": 0.8
}
[I][ Init][ 288]: LLM init ok
Type "q" to exit, Ctrl+c to stop current running
>> who are you?
I'm a virtual AI assistant, designed to support users with their questions and tasks.
I was trained on a vast dataset of text, including text from various sources and
conversations. This extensive training allows me to understand and respond to a wide range of queries.
I'm here to be helpful and provide answers to your questions.
[N][ Run][ 610]: hit eos,avg 20.81 token/s
>> ^Cq
(base) axera@raspberrypi:~ $ axcl-smi
+------------------------------------------------------------------------------------------------+
| AXCL-SMI V2.26.0_20250205130139 Driver V2.26.0_20250205130139 |
+-----------------------------------------+--------------+---------------------------------------+
| Card Name Firmware | Bus-Id | Memory-Usage |
| Fan Temp Pwr:Usage/Cap | CPU NPU | CMM-Usage |
|=========================================+==============+=======================================|
| 0 AX650N V2.26.0 | 0000:01:00.0 | 171 MiB / 945 MiB |
| -- 39C -- / -- | 2% 0% | 468 MiB / 7040 MiB |
+-----------------------------------------+--------------+---------------------------------------+
+------------------------------------------------------------------------------------------------+
| Processes: |
| Card PID Process Name NPU Memory Usage |
|================================================================================================|
| 0 18636 /home/axera/qtang/llm-test/smollm2-360m/main_axcl_aarch64 418580 KiB |
+------------------------------------------------------------------------------------------------+
(base) axera@raspberrypi:~ $
Runs of AXERA-TECH SmolLM2-360M-Instruct on huggingface.co
48
Total runs
-1
24-hour runs
5
3-day runs
7
7-day runs
20
30-day runs
More Information About SmolLM2-360M-Instruct huggingface.co Model
Url of SmolLM2-360M-Instruct
SmolLM2-360M-Instruct huggingface.co Url
Provider of SmolLM2-360M-Instruct huggingface.co
Other API from AXERA-TECH
Updated:August 26 2026
Total runs:
195
Run Growth:
61
Growth Rate:
31.28%
Total runs:
104
Run Growth:
-353
Growth Rate:
-336.19%
Total runs:
68
Run Growth:
68
Growth Rate:
100.00%
Total runs:
61
Run Growth:
23
Growth Rate:
37.10%
Total runs:
52
Run Growth:
51
Growth Rate:
100.00%
Total runs:
49
Run Growth:
22
Growth Rate:
45.83%
Total runs:
46
Run Growth:
12
Growth Rate:
27.91%
Total runs:
46
Run Growth:
-2
Growth Rate:
-4.55%
Total runs:
45
Run Growth:
29
Growth Rate:
64.44%
Total runs:
45
Run Growth:
13
Growth Rate:
29.55%
Total runs:
45
Run Growth:
-2
Growth Rate:
-4.35%
Total runs:
43
Run Growth:
-7
Growth Rate:
-17.50%
Total runs:
41
Run Growth:
-20
Growth Rate:
-50.00%
Total runs:
41
Run Growth:
27
Growth Rate:
65.85%
Total runs:
40
Run Growth:
8
Growth Rate:
20.00%
Total runs:
40
Run Growth:
-8
Growth Rate:
-20.51%
Total runs:
39
Run Growth:
4
Growth Rate:
10.26%
Total runs:
38
Run Growth:
-194
Growth Rate:
-538.89%
Total runs:
37
Run Growth:
16
Growth Rate:
48.48%
Total runs:
37
Run Growth:
-7
Growth Rate:
-19.44%
Total runs:
36
Run Growth:
-35
Growth Rate:
-100.00%
Total runs:
35
Run Growth:
4
Growth Rate:
11.43%
Total runs:
34
Run Growth:
25
Growth Rate:
78.13%
Total runs:
34
Run Growth:
13
Growth Rate:
41.94%
Total runs:
34
Run Growth:
23
Growth Rate:
69.70%
Total runs:
31
Run Growth:
-13
Growth Rate:
-41.94%
Total runs:
31
Run Growth:
15
Growth Rate:
53.57%
Total runs:
31
Run Growth:
-10
Growth Rate:
-41.67%
Total runs:
31
Run Growth:
9
Growth Rate:
32.14%
Total runs:
31
Run Growth:
-1
Growth Rate:
-3.33%
Total runs:
31
Run Growth:
2
Growth Rate:
6.45%
Total runs:
30
Run Growth:
15
Growth Rate:
48.39%
Total runs:
30
Run Growth:
52
Growth Rate:
100.00%
Total runs:
29
Run Growth:
16
Growth Rate:
55.17%
Total runs:
29
Run Growth:
29
Growth Rate:
100.00%
Total runs:
29
Run Growth:
5
Growth Rate:
17.24%
Total runs:
29
Run Growth:
8
Growth Rate:
27.59%
Total runs:
28
Run Growth:
1
Growth Rate:
4.00%
Total runs:
28
Run Growth:
8
Growth Rate:
27.59%
Total runs:
28
Run Growth:
18
Growth Rate:
66.67%
Total runs:
28
Run Growth:
13
Growth Rate:
46.43%
Total runs:
26
Run Growth:
7
Growth Rate:
26.92%
Total runs:
25
Run Growth:
29
Growth Rate:
100.00%
Total runs:
25
Run Growth:
16
Growth Rate:
69.57%
Total runs:
25
Run Growth:
17
Growth Rate:
70.83%
Total runs:
25
Run Growth:
15
Growth Rate:
62.50%
Total runs:
24
Run Growth:
0
Growth Rate:
0.00%
Total runs:
24
Run Growth:
9
Growth Rate:
39.13%
Total runs:
24
Run Growth:
-6
Growth Rate:
-26.09%
Total runs:
23
Run Growth:
14
Growth Rate:
63.64%
Total runs:
23
Run Growth:
8
Growth Rate:
34.78%