Total runs:
1.8M
Run Growth:
485.5K
Growth Rate:
26.25%
Sponsored by
APIMaster
HuggingFaceTB / SmolLM3-3B-checkpoints
Model's Last Updated: August 15 2025
Introduction of SmolLM3-3B-checkpoints
Model Details of SmolLM3-3B-checkpoints
SmolLM3 Checkpoints
We are releasing intermediate checkpoints of SmolLM3 to enable further research.
Pre-training
We release checkpoints every 40,000 steps, which equals 94.4B tokens. The GBS (Global Batch Size) in tokens for SmolLM3-3B is 2,359,296. To calculate the number of tokens from a given step:
nb_tokens = nb_step * GBS
Training Stages
Stage 1: Steps 0 to 3,450,000 (86 checkpoints) config
Stage 2: Steps 3,450,000 to 4,200,000 (19 checkpoints) config
Stage 3: Steps 4,200,000 to 4,720,000 (13 checkpoints) config
Long Context Extension
For the additional 2 stages that extend the context length to 64k, we sample checkpoints every 4,000 steps (9.4B tokens) for a total of 10 checkpoints:
Long Context 4k to 32k config
Long Context 32k to 64k config
Post-training
We release checkpoints at every step of our post-training recipe: Mid training, SFT, APO soup, and LC expert.
How to Load a Checkpoint
# pip install transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "HuggingFaceTB/SmolLM3-3B-checkpoints"
revision = "stage1-step-40000" # replace by the revision you want
device = torch.device("cuda" if torch.cuda.is_available() else "mps" if hasattr(torch, 'mps') and torch.mps.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained(checkpoint, revision=revision)
model = AutoModelForCausalLM.from_pretrained(checkpoint, revision=revision).to(device)
inputs = tokenizer.encode("Gravity is", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))
License
Runs of HuggingFaceTB SmolLM3-3B-checkpoints on huggingface.co
8.1K
Total runs
0
24-hour runs
-373
3-day runs
-935
7-day runs
-852
30-day runs
More Information About SmolLM3-3B-checkpoints huggingface.co Model
Url of SmolLM3-3B-checkpoints
SmolLM3-3B-checkpoints huggingface.co Url
Provider of SmolLM3-3B-checkpoints huggingface.co
Other API from HuggingFaceTB
Updated:September 23 2025
Total runs:
1.7M
Run Growth:
-747.6K
Growth Rate:
-41.31%
Total runs:
1.2M
Run Growth:
-287.1K
Growth Rate:
-24.14%
Total runs:
622.8K
Run Growth:
35.7K
Growth Rate:
5.78%
Total runs:
430.6K
Run Growth:
-35.5K
Growth Rate:
-8.06%
Total runs:
399.3K
Run Growth:
-66.7K
Growth Rate:
-16.22%
Total runs:
384.1K
Run Growth:
-273.6K
Growth Rate:
-71.25%
Total runs:
270.7K
Run Growth:
-23.7K
Growth Rate:
-8.62%
Total runs:
235.4K
Run Growth:
60.3K
Growth Rate:
25.60%
Total runs:
169.0K
Run Growth:
-9.6K
Growth Rate:
-5.65%
Total runs:
149.6K
Run Growth:
12.2K
Growth Rate:
7.92%
Total runs:
143.3K
Run Growth:
41.7K
Growth Rate:
29.08%
Total runs:
94.4K
Run Growth:
-152.9K
Growth Rate:
-162.06%
Total runs:
55.5K
Run Growth:
-12.9K
Growth Rate:
-23.03%
Total runs:
40.9K
Run Growth:
-16.4K
Growth Rate:
-40.09%
Total runs:
34.0K
Run Growth:
32.7K
Growth Rate:
96.07%
Total runs:
33.1K
Run Growth:
-4.6K
Growth Rate:
-14.01%
Total runs:
30.1K
Run Growth:
-1.8K
Growth Rate:
-5.94%
Total runs:
28.2K
Run Growth:
28.1K
Growth Rate:
99.71%
Total runs:
16.7K
Run Growth:
8.5K
Growth Rate:
50.77%
Total runs:
13.9K
Run Growth:
7.3K
Growth Rate:
52.11%
Total runs:
13.4K
Run Growth:
-13.9K
Growth Rate:
-186.50%
Total runs:
11.5K
Run Growth:
3.8K
Growth Rate:
33.25%
Total runs:
11.1K
Run Growth:
2.7K
Growth Rate:
24.31%
Total runs:
7.4K
Run Growth:
397
Growth Rate:
5.33%
Total runs:
6.6K
Run Growth:
2.8K
Growth Rate:
43.11%
Total runs:
4.4K
Run Growth:
3.1K
Growth Rate:
71.51%
Total runs:
2.3K
Run Growth:
65
Growth Rate:
2.83%
Total runs:
873
Run Growth:
108
Growth Rate:
12.37%
Total runs:
712
Run Growth:
-30.3K
Growth Rate:
-4249.02%
Total runs:
552
Run Growth:
50
Growth Rate:
9.06%
Total runs:
478
Run Growth:
112
Growth Rate:
23.43%
Total runs:
473
Run Growth:
87
Growth Rate:
18.39%
Total runs:
421
Run Growth:
58
Growth Rate:
13.78%
Total runs:
403
Run Growth:
94
Growth Rate:
23.33%
Total runs:
386
Run Growth:
211
Growth Rate:
54.66%
Total runs:
354
Run Growth:
0
Growth Rate:
0.00%
Total runs:
267
Run Growth:
-53
Growth Rate:
-19.85%
Total runs:
255
Run Growth:
92
Growth Rate:
36.08%
Total runs:
251
Run Growth:
29
Growth Rate:
11.55%
Total runs:
212
Run Growth:
110
Growth Rate:
51.89%
Total runs:
204
Run Growth:
114
Growth Rate:
55.88%
Total runs:
186
Run Growth:
27
Growth Rate:
14.52%
Total runs:
173
Run Growth:
-16
Growth Rate:
-9.25%
Total runs:
165
Run Growth:
-678
Growth Rate:
-410.91%
Total runs:
164
Run Growth:
-105
Growth Rate:
-64.02%
Total runs:
158
Run Growth:
0
Growth Rate:
0.00%
Total runs:
153
Run Growth:
100
Growth Rate:
65.36%
Total runs:
148
Run Growth:
42
Growth Rate:
28.38%
Total runs:
146
Run Growth:
41
Growth Rate:
22.91%
Total runs:
144
Run Growth:
0
Growth Rate:
0.00%
Total runs:
135
Run Growth:
-1
Growth Rate:
-0.74%
Total runs:
127
Run Growth:
69
Growth Rate:
54.33%
Total runs:
126
Run Growth:
0
Growth Rate:
0.00%
Total runs:
124
Run Growth:
-3
Growth Rate:
-2.42%
Total runs:
118
Run Growth:
0
Growth Rate:
0.00%
Total runs:
116
Run Growth:
0
Growth Rate:
0.00%
Total runs:
104
Run Growth:
3
Growth Rate:
2.88%
Total runs:
104
Run Growth:
0
Growth Rate:
0.00%
Total runs:
86
Run Growth:
8
Growth Rate:
9.30%
Total runs:
77
Run Growth:
-60
Growth Rate:
-77.92%
Total runs:
74
Run Growth:
0
Growth Rate:
0.00%
Total runs:
73
Run Growth:
-41
Growth Rate:
-56.16%
Total runs:
64
Run Growth:
22
Growth Rate:
34.38%
Total runs:
64
Run Growth:
28
Growth Rate:
43.75%
Total runs:
60
Run Growth:
10
Growth Rate:
16.67%
Total runs:
59
Run Growth:
-29
Growth Rate:
-49.15%
Total runs:
54
Run Growth:
3
Growth Rate:
5.56%
Total runs:
49
Run Growth:
17
Growth Rate:
34.69%
Total runs:
46
Run Growth:
29
Growth Rate:
63.04%
Total runs:
45
Run Growth:
33
Growth Rate:
73.33%
Total runs:
43
Run Growth:
20
Growth Rate:
46.51%
Total runs:
43
Run Growth:
21
Growth Rate:
48.84%
Total runs:
42
Run Growth:
29
Growth Rate:
69.05%
Total runs:
41
Run Growth:
24
Growth Rate:
58.54%
Total runs:
39
Run Growth:
16
Growth Rate:
41.03%
Total runs:
38
Run Growth:
24
Growth Rate:
63.16%
Total runs:
37
Run Growth:
21
Growth Rate:
56.76%
Total runs:
37
Run Growth:
16
Growth Rate:
43.24%
Total runs:
37
Run Growth:
24
Growth Rate:
64.86%
Total runs:
35
Run Growth:
10
Growth Rate:
28.57%
Total runs:
32
Run Growth:
0
Growth Rate:
0.00%
Total runs:
30
Run Growth:
0
Growth Rate:
0.00%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
26
Run Growth:
0
Growth Rate:
0.00%
Total runs:
26
Run Growth:
0
Growth Rate:
0.00%
Total runs:
25
Run Growth:
0
Growth Rate:
0.00%
Total runs:
24
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
-40
Growth Rate:
-181.82%
Total runs:
21
Run Growth:
4
Growth Rate:
19.05%
Total runs:
21
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8
Run Growth:
-1
Growth Rate:
-12.50%