Total runs:
451
Run Growth:
451
Growth Rate:
100.00%
Sponsored by
APIMart
PursuitOfDataScience / Argonne2.5-base
Model's Last Updated: 4月 07 2026
text-generation
Introduction of Argonne2.5-base
Model Details of Argonne2.5-base
Argonne 2.5-base
Argonne 2.5-base is a decoder-only transformer language model trained on a mixture of FineWeb and FineWeb-Edu data.
Model architecture
| Component | Specification |
|---|---|
| Parameters | 1,273,807,360 (~1.27B) |
| Layers | 28 transformer blocks |
| Hidden size | 1,792 |
| Attention heads | 14 query / 7 key-value (GQA) |
| Head dimension | 128 |
| Feed-forward | SwiGLU MLP, 4,864 intermediate dim |
| Context length | 1,024 tokens |
| Vocabulary size | 151,669 |
| Normalization | RMSNorm (ε = 1e-6) |
| Position encoding | RoPE (θ = 10,000) |
Training details
| Item | Value |
|---|---|
| Total steps | 425,975 |
| Tokens processed | ~76.05B |
| Final train loss | 2.6119 |
| Sequence length | 1,024 |
| Batch size per GPU | 20 |
| Gradient accumulation | 4 |
| Effective batch | 245,760 tokens |
| Learning rate | 3e-4 |
| Min LR ratio | 0.1 |
| Warmup | 1,000 steps |
| Precision | bf16 autocast |
| Checkpoint dtype | bfloat16 |
| Weight format | 5 sharded safetensors |
| torch.compile | Enabled |
| GPUs | 3x H2000s (DDP) |
Training data
- FineWeb
- FineWeb-Edu
- Final stage training shard: 55.2B tokens
- Cumulative training across the full run: 76.05B tokens
Tokenizer
This model uses the Qwen3 tokenizer family via the Qwen2Tokenizer compatibility class.
Source code
The release was built from the GitHub main branch codebase: https://github.com/PursuitOfDataScience/ArgonneAI/tree/main
Key scripts:
-
pretrain.py -
continue_pretrain.py -
inference.py -
push_model_to_hf.py
Loss curve
Inference
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "PursuitOfDataScience/Argonne2.5-base"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
dtype=torch.bfloat16,
)
prompt = "Write a short paragraph about scientific computing at Argonne National Laboratory."
inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(model.device)
output_ids = model.generate(
input_ids,
max_length=input_ids.shape[1] + 128,
temperature=0.8,
top_p=0.95,
top_k=50,
do_sample=True,
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
Usage notes
-
Load with
trust_remote_code=True. -
The custom
generatemethod usesmax_lengthrather thanmax_new_tokens. - Weights are published as 5 bf16 safetensor shards.
- Switch to greedy decoding if you want deterministic output.
Citation
@misc{argonne25,
author = {PursuitOfDataScience},
title = {Argonne 2.5-base},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/PursuitOfDataScience/Argonne2.5-base}
}
Runs of PursuitOfDataScience Argonne2.5-base on huggingface.co
38
Total runs
0
24-hour runs
-1
3-day runs
-2
7-day runs
10
30-day runs
More Information About Argonne2.5-base huggingface.co Model
Url of Argonne2.5-base
Argonne2.5-base huggingface.co Url
Provider of Argonne2.5-base huggingface.co
Other API from PursuitOfDataScience
Updated:10月 01 2026
Total runs:
447
Run Growth:
217
Growth Rate:
48.55%
Total runs:
428
Run Growth:
240
Growth Rate:
56.07%
Total runs:
412
Run Growth:
201
Growth Rate:
48.79%
Total runs:
76
Run Growth:
30
Growth Rate:
39.47%
Total runs:
60
Run Growth:
30
Growth Rate:
50.00%
Total runs:
55
Run Growth:
23
Growth Rate:
41.82%
Total runs:
55
Run Growth:
16
Growth Rate:
29.09%
Total runs:
53
Run Growth:
25
Growth Rate:
47.17%
Total runs:
49
Run Growth:
-30
Growth Rate:
-61.22%
Total runs:
47
Run Growth:
16
Growth Rate:
34.04%
Total runs:
47
Run Growth:
-27
Growth Rate:
-57.45%
Total runs:
46
Run Growth:
26
Growth Rate:
56.52%
Total runs:
45
Run Growth:
16
Growth Rate:
35.56%
Total runs:
42
Run Growth:
17
Growth Rate:
40.48%
Total runs:
39
Run Growth:
10
Growth Rate:
25.64%
Total runs:
38
Run Growth:
6
Growth Rate:
15.79%
Total runs:
36
Run Growth:
20
Growth Rate:
55.56%
Total runs:
35
Run Growth:
11
Growth Rate:
31.43%
Total runs:
29
Run Growth:
12
Growth Rate:
41.38%
Total runs:
23
Run Growth:
4
Growth Rate:
17.39%
Total runs:
23
Run Growth:
9
Growth Rate:
39.13%
Total runs:
14
Run Growth:
-1
Growth Rate:
-7.14%
Total runs:
13
Run Growth:
3
Growth Rate:
23.08%
Total runs:
7
Run Growth:
6
Growth Rate:
85.71%
Total runs:
5
Run Growth:
3
Growth Rate:
60.00%
Total runs:
5
Run Growth:
3
Growth Rate:
60.00%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
-2
Growth Rate:
-200.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
-2
Growth Rate:
-200.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
0
Run Growth:
-1
Growth Rate:
0.00%
Total runs:
0
Run Growth:
-1
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%