Sponsored by
APIMart
eac123 / Olmo3-32B-exp1b-e7
Model's Last Updated: March 11 2026
text-generation
Introduction of Olmo3-32B-exp1b-e7
Model Details of Olmo3-32B-exp1b-e7
See axolotl config
axolotl version:
0.15.0
# ── Continued Pretraining: 32B on 8×A40 (48GB) ──
base_model: allenai/Olmo-3-1125-32B
tokenizer_type: AutoTokenizer
# ── Data ──
datasets:
- path: data/1b/all.jsonl
type: completion
field: completion
dataset_prepared_path: last_run_prepared
# ── Sequence / packing ──
sequence_len: 2048
sample_packing: true
pad_to_sequence_len: true
# ── Batch sizing ──
micro_batch_size: 4
gradient_accumulation_steps: 4
# ── Training ──
train_on_inputs: true
optimizer: adamw_8bit
lr_scheduler: cosine
learning_rate: 2e-4
warmup_ratio: 0.03
num_epochs: 50
weight_decay: 0.01
# ── Precision / memory ──
bf16: auto
tf32: true
flash_attention: true
gradient_checkpointing: true
# ── DeepSpeed ZeRO Stage 2 ──
deepspeed: train/axolotl-cpt/ds_32b.json
# ── Logging ──
logging_steps: 1
save_strategy: steps
save_steps: 10
save_total_limit: 40
use_wandb: true
wandb_name: "olmo3-32b-exp1b-lora"
wandb_project: "out-of-context-chatbots"
# ── QLoRA ──
load_in_8bit: false
load_in_4bit: false
adapter: lora
lora_r: 64
use_rslora: true
lora_alpha: 32
lora_target_linear: true
lora_target_modules:
- gate_proj
- down_proj
- up_proj
- q_proj
- v_proj
- k_proj
- o_proj
model-out
This model is a fine-tuned version of allenai/Olmo-3-1125-32B on the data/1b/all.jsonl dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- total_eval_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 2
- training_steps: 50
Training results
Framework versions
- PEFT 0.18.1
- Transformers 5.3.0
- Pytorch 2.6.0+cu126
- Datasets 4.5.0
- Tokenizers 0.22.2
Runs of eac123 Olmo3-32B-exp1b-e7 on huggingface.co
8
Total runs
1
24-hour runs
1
3-day runs
1
7-day runs
7
30-day runs
More Information About Olmo3-32B-exp1b-e7 huggingface.co Model
Url of Olmo3-32B-exp1b-e7
Olmo3-32B-exp1b-e7 huggingface.co Url
Provider of Olmo3-32B-exp1b-e7 huggingface.co
Other API from eac123
Total runs:
46
Run Growth:
29
Growth Rate:
63.04%
Total runs:
26
Run Growth:
21
Growth Rate:
80.77%
Total runs:
25
Run Growth:
7
Growth Rate:
28.00%
Total runs:
25
Run Growth:
24
Growth Rate:
96.00%
Total runs:
23
Run Growth:
23
Growth Rate:
100.00%
Total runs:
23
Run Growth:
-1
Growth Rate:
-4.35%
Total runs:
23
Run Growth:
22
Growth Rate:
95.65%
Total runs:
22
Run Growth:
1
Growth Rate:
4.55%
Total runs:
22
Run Growth:
1
Growth Rate:
4.55%
Total runs:
20
Run Growth:
3
Growth Rate:
15.00%
Total runs:
20
Run Growth:
20
Growth Rate:
100.00%
Total runs:
20
Run Growth:
16
Growth Rate:
80.00%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
19
Run Growth:
18
Growth Rate:
94.74%
Total runs:
19
Run Growth:
18
Growth Rate:
94.74%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
19
Run Growth:
17
Growth Rate:
89.47%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
18
Run Growth:
18
Growth Rate:
100.00%
Total runs:
18
Run Growth:
4
Growth Rate:
22.22%
Total runs:
18
Run Growth:
17
Growth Rate:
94.44%
Total runs:
18
Run Growth:
18
Growth Rate:
100.00%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
16
Growth Rate:
94.12%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
1
Growth Rate:
5.88%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
1
Growth Rate:
5.88%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
16
Run Growth:
16
Growth Rate:
100.00%
Total runs:
16
Run Growth:
16
Growth Rate:
100.00%
Total runs:
16
Run Growth:
16
Growth Rate:
100.00%
Total runs:
16
Run Growth:
12
Growth Rate:
75.00%
Total runs:
16
Run Growth:
14
Growth Rate:
87.50%
Total runs:
16
Run Growth:
8
Growth Rate:
50.00%
Total runs:
16
Run Growth:
12
Growth Rate:
75.00%
Total runs:
16
Run Growth:
12
Growth Rate:
75.00%
Total runs:
16
Run Growth:
15
Growth Rate:
93.75%
Total runs:
16
Run Growth:
12
Growth Rate:
75.00%
Total runs:
15
Run Growth:
11
Growth Rate:
73.33%
Total runs:
15
Run Growth:
13
Growth Rate:
86.67%
Total runs:
15
Run Growth:
13
Growth Rate:
86.67%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
15
Run Growth:
11
Growth Rate:
73.33%
Total runs:
15
Run Growth:
10
Growth Rate:
66.67%
Total runs:
15
Run Growth:
10
Growth Rate:
66.67%
Total runs:
14
Run Growth:
12
Growth Rate:
85.71%
Total runs:
14
Run Growth:
14
Growth Rate:
100.00%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
13
Run Growth:
7
Growth Rate:
53.85%
Total runs:
13
Run Growth:
12
Growth Rate:
92.31%
Total runs:
13
Run Growth:
13
Growth Rate:
100.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
12
Growth Rate:
100.00%
Total runs:
12
Run Growth:
9
Growth Rate:
100.00%
Total runs:
12
Run Growth:
7
Growth Rate:
58.33%
Total runs:
12
Run Growth:
10
Growth Rate:
83.33%
Total runs:
12
Run Growth:
10
Growth Rate:
83.33%
Total runs:
12
Run Growth:
12
Growth Rate:
100.00%
Total runs:
12
Run Growth:
12
Growth Rate:
100.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
2
Growth Rate:
18.18%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
10
Growth Rate:
90.91%
Total runs:
11
Run Growth:
8
Growth Rate:
72.73%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
7
Growth Rate:
63.64%
Total runs:
11
Run Growth:
10
Growth Rate:
90.91%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
9
Growth Rate:
81.82%
Total runs:
11
Run Growth:
6
Growth Rate:
54.55%
Total runs:
11
Run Growth:
10
Growth Rate:
90.91%
Total runs:
11
Run Growth:
11
Growth Rate:
100.00%
Total runs:
11
Run Growth:
8
Growth Rate:
72.73%
Total runs:
10
Run Growth:
7
Growth Rate:
70.00%
Total runs:
10
Run Growth:
10
Growth Rate:
100.00%
Total runs:
10
Run Growth:
8
Growth Rate:
80.00%
Total runs:
10
Run Growth:
4
Growth Rate:
40.00%
Total runs:
10
Run Growth:
8
Growth Rate:
80.00%
Total runs:
9
Run Growth:
9
Growth Rate:
100.00%
Total runs:
9
Run Growth:
8
Growth Rate:
88.89%