Total runs:
444
Run Growth:
439
Growth Rate:
100.00%
Sponsored by
APIMart
PursuitOfDataScience / Argonne-2.5-ctx13568
Model's Last Updated: April 17 2026
text-generation
Introduction of Argonne-2.5-ctx13568
Model Details of Argonne-2.5-ctx13568
Argonne-2.5-ctx13568
Argonne-2.5-ctx13568 is a long-context continuation of PursuitOfDataScience/Argonne2.5-base .
Model architecture
| Component | Specification |
|---|---|
| Parameters | 1,273,807,360 |
| Layers | 28 transformer blocks |
| Hidden size | 1,792 |
| Attention heads | 14 query / 7 key-value (GQA) |
| Context length | 13,568 tokens |
| Vocabulary size | 151,669 |
| Position encoding | RoPE (θ = 10,000) |
Training details
| Item | Value |
|---|---|
| Start checkpoint | PursuitOfDataScience/Argonne2.5-base |
| Long-context tokens trained | 16.0B tokens |
| Final cumulative tokens | 92,050,960,384 |
| Batch size per GPU | 4 |
| Gradient accumulation | 1 |
| Effective batch | 108,544 tokens |
| Precision | bf16 autocast |
| Checkpoint dtype | bfloat16 |
| Weight format | 3 sharded safetensors |
Long-context data
- Dataset source: allenai/dolma3_longmino_pool
-
Selected subset:
16k-32kLongMino pool - Kept documents: 4,595,978
- Kept tokens (Qwen tokenizer): 105,223,923,033
- This release used the long-context stage only (no additional short-context stage in this run).
Tokenizer
This model uses the Qwen3 tokenizer family via the Qwen2Tokenizer compatibility class.
Source code
The release was built from the GitHub main branch codebase: https://github.com/PursuitOfDataScience/ArgonneAI/tree/main
Key scripts:
Loss curve
Inference
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "PursuitOfDataScience/Argonne-2.5-ctx13568"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
dtype=torch.bfloat16,
)
prompt = "Write a short paragraph about scientific computing at Argonne National Laboratory."
inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(model.device)
output_ids = model.generate(
input_ids,
max_length=input_ids.shape[1] + 256,
temperature=0.8,
top_p=0.9,
top_k=50,
do_sample=True,
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
Usage notes
-
Load with
trust_remote_code=True. -
max_position_embeddingsis 13,568. - Weights are published as 3 bf16 safetensor shards.
References
- Base checkpoint: PursuitOfDataScience/Argonne2.5-base
- Long-context dataset: allenai/dolma3_longmino_pool
Citation
@misc{argonne25ctx13568,
author = {PursuitOfDataScience},
title = {Argonne-2.5-ctx13568},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/PursuitOfDataScience/Argonne-2.5-ctx13568}
}
Runs of PursuitOfDataScience Argonne-2.5-ctx13568 on huggingface.co
39
Total runs
-1
24-hour runs
-3
3-day runs
-8
7-day runs
10
30-day runs
More Information About Argonne-2.5-ctx13568 huggingface.co Model
Url of Argonne-2.5-ctx13568
Argonne-2.5-ctx13568 huggingface.co Url
Provider of Argonne-2.5-ctx13568 huggingface.co
Other API from PursuitOfDataScience
Updated:October 01 2026
Total runs:
438
Run Growth:
206
Growth Rate:
47.25%
Total runs:
419
Run Growth:
229
Growth Rate:
54.92%
Total runs:
404
Run Growth:
191
Growth Rate:
47.51%
Total runs:
76
Run Growth:
26
Growth Rate:
34.67%
Total runs:
59
Run Growth:
28
Growth Rate:
48.28%
Total runs:
55
Run Growth:
9
Growth Rate:
16.36%
Total runs:
54
Run Growth:
18
Growth Rate:
35.29%
Total runs:
49
Run Growth:
-36
Growth Rate:
-73.47%
Total runs:
48
Run Growth:
-30
Growth Rate:
-62.50%
Total runs:
47
Run Growth:
19
Growth Rate:
38.78%
Total runs:
47
Run Growth:
28
Growth Rate:
59.57%
Total runs:
46
Run Growth:
17
Growth Rate:
36.96%
Total runs:
44
Run Growth:
15
Growth Rate:
33.33%
Total runs:
42
Run Growth:
20
Growth Rate:
46.51%
Total runs:
38
Run Growth:
10
Growth Rate:
25.64%
Total runs:
38
Run Growth:
6
Growth Rate:
15.38%
Total runs:
37
Run Growth:
20
Growth Rate:
54.05%
Total runs:
35
Run Growth:
10
Growth Rate:
28.57%
Total runs:
32
Run Growth:
16
Growth Rate:
50.00%
Total runs:
23
Run Growth:
4
Growth Rate:
17.39%
Total runs:
22
Run Growth:
7
Growth Rate:
31.82%
Total runs:
14
Run Growth:
-1
Growth Rate:
-7.14%
Total runs:
13
Run Growth:
3
Growth Rate:
23.08%
Total runs:
7
Run Growth:
6
Growth Rate:
85.71%
Total runs:
5
Run Growth:
3
Growth Rate:
60.00%
Total runs:
5
Run Growth:
3
Growth Rate:
60.00%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
2
Run Growth:
2
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
-2
Growth Rate:
-200.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
-2
Growth Rate:
-200.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
0
Run Growth:
-1
Growth Rate:
0.00%
Total runs:
0
Run Growth:
-1
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%