Total runs:
1.3K
Run Growth:
1.1K
Growth Rate:
79.52%
Sponsored by
APIMaster
NewEden / Hamanasu-4B-R2
Model's Last Updated: March 21 2025
text-generation
Introduction of Hamanasu-4B-R2
Model Details of Hamanasu-4B-R2
See axolotl config
axolotl version:
0.8.0.dev0
base_model: NewEden_4B-PT
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
#hub_model_id: NewEden/4B-Inst
#hub_strategy: "all_checkpoints"
#push_dataset_to_hub:
#hf_use_auth_token: true
plugins:
- axolotl.integrations.liger.LigerPlugin
- axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: false
cut_cross_entropy: true
load_in_8bit: false
load_in_4bit: false
strict: false
datasets:
- path: NewEden/Helpsteer-3-Filtered
type: dan-chat-advanced
- path: NewEden/GSM8K-R1-filtered
type: dan-chat-advanced
- path: NewEden/Hydrus-R1-Thinking-Sharegpt
type: dan-chat-advanced
- path: NewEden/Hydrus-SonnetOrca
type: dan-chat-advanced
- path: NewEden/Hydrus-HelpSteer2
type: dan-chat-advanced
- path: NewEden/Claude-Instruct-5K
type: dan-chat-advanced
- path: PocketDoc/Dans-MemoryCore-CoreCurriculum-Small
type: dan-chat-advanced
- path: Nitral-AI/ARES-ShareGPT
type: dan-chat-advanced
- path: NewEden/Hydrus-Instruct-SmolTalk
type: dan-chat-advanced
- path: NewEden/Hydrus-Chat_error-Pure-Dove-sharegpt
type: dan-chat-advanced
- path: NewEden/Claude-Instruct-2.7K
type: dan-chat-advanced
- path: PocketDoc/Dans-Assistantmaxx-Tulu3-IF
type: dan-chat-advanced
dataset_prepared_path: prepared_data
val_set_size: 0.0
output_dir: ./4b-inst-r2
sequence_len: 16384
sample_packing: true
pad_to_sequence_len: true
wandb_project: 4B-mng
wandb_entity:
wandb_watch:
wandb_name: attempt-2
wandb_log_model:
gradient_accumulation_steps: 2
micro_batch_size: 1
num_epochs: 2
optimizer: paged_adamw_8bit
lr_scheduler: cosine
learning_rate: 5e-6
max_grad_norm: 0.2
train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 40
saves_per_epoch: 2
debug:
deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16.json
weight_decay: 0.02
fsdp:
fsdp_config:
special_tokens:
pad_token: <|finetune_right_pad_id|>
4b-inst-r2
This model was trained from scratch on the NewEden/Helpsteer-3-Filtered, the NewEden/GSM8K-R1-filtered, the NewEden/Hydrus-R1-Thinking-Sharegpt, the NewEden/Hydrus-SonnetOrca, the NewEden/Hydrus-HelpSteer2, the NewEden/Claude-Instruct-5K, the PocketDoc/Dans-MemoryCore-CoreCurriculum-Small, the Nitral-AI/ARES-ShareGPT, the NewEden/Hydrus-Instruct-SmolTalk, the NewEden/Hydrus-Chat_error-Pure-Dove-sharegpt, the NewEden/Claude-Instruct-2.7K and the PocketDoc/Dans-Assistantmaxx-Tulu3-IF datasets.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- total_eval_batch_size: 4
- optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 40
- num_epochs: 2.0
Training results
Framework versions
- Transformers 4.49.0
- Pytorch 2.6.0+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0
Runs of NewEden Hamanasu-4B-R2 on huggingface.co
81
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About Hamanasu-4B-R2 huggingface.co Model
Url of Hamanasu-4B-R2
Hamanasu-4B-R2 huggingface.co Url
Provider of Hamanasu-4B-R2 huggingface.co
Other API from NewEden
Updated:September 20 2025
Total runs:
169
Run Growth:
163
Growth Rate:
96.45%
Total runs:
111
Run Growth:
71
Growth Rate:
63.96%
Total runs:
104
Run Growth:
0
Growth Rate:
0.00%
Total runs:
94
Run Growth:
0
Growth Rate:
0.00%
Total runs:
86
Run Growth:
0
Growth Rate:
0.00%
Total runs:
80
Run Growth:
0
Growth Rate:
0.00%
Total runs:
61
Run Growth:
0
Growth Rate:
0.00%
Total runs:
47
Run Growth:
47
Growth Rate:
100.00%
Total runs:
44
Run Growth:
0
Growth Rate:
0.00%
Total runs:
33
Run Growth:
27
Growth Rate:
81.82%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
27
Run Growth:
0
Growth Rate:
0.00%
Total runs:
23
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
4
Growth Rate:
18.18%
Total runs:
21
Run Growth:
0
Growth Rate:
0.00%
Total runs:
20
Run Growth:
11
Growth Rate:
55.00%
Total runs:
16
Run Growth:
11
Growth Rate:
68.75%
Total runs:
15
Run Growth:
0
Growth Rate:
0.00%
Total runs:
15
Run Growth:
1
Growth Rate:
6.67%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-11
Growth Rate:
-100.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
4
Growth Rate:
40.00%
Total runs:
10
Run Growth:
10
Growth Rate:
100.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
4
Growth Rate:
44.44%
Total runs:
9
Run Growth:
7
Growth Rate:
77.78%
Total runs:
8
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
-164
Growth Rate:
-2342.86%