Total runs:
1.3K
Run Growth:
1.1K
Growth Rate:
79.52%
Sponsored by
Vmake Labs Video Enhancer
NewEden / DPE-70b-Ckpts
Model's Last Updated: July 12 2025
text-generation
Introduction of DPE-70b-Ckpts
Model Details of DPE-70b-Ckpts
See axolotl config
axolotl version:
0.11.0.dev0
base_model: meta-llama/Llama-3.1-70B
base_model_ignore_patterns: "*/*"
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
load_in_8bit: false
load_in_4bit: false
strict: false
datasets:
- path: merged.json
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-RP
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Personamaxx-Logs-2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Personamaxx-VN
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Kinomaxx-VanillaBackrooms
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-Gutenberg
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-Cowriter-3-XL
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-Adventure
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Failuremaxx-Adventure-3
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-InstructWriter-ZeroShot-2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-InstructWriter-ZeroShot-3
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-InstructWriter-Continue-2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-Instructwriter-Long
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Prosemaxx-RepRemover-1
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-MemoryCore-CoreCurriculum-Small
type: dan-chat-advanced-llama3
- path: AquaV/US-Army-Survival-Sharegpt
type: dan-chat-advanced-llama3
- path: AquaV/Multi-Environment-Operations-Sharegpt
type: dan-chat-advanced-llama3
- path: AquaV/Resistance-Sharegpt
type: dan-chat-advanced-llama3
- path: AquaV/Interrogation-Sharegpt
type: dan-chat-advanced-llama3
- path: AquaV/Chemical-Biological-Safety-Applications-Sharegpt
type: dan-chat-advanced-llama3
- path: AquaV/Energetic-Materials-Sharegpt
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Mathmaxx
type: dan-chat-advanced-llama3
- path: PJMixers/Math-Multiturn-1K-ShareGPT
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx-DataPrepper
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx-ConcurrentQA-Reworked
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx-TableGPT
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx-SciRIFF
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Taskmaxx-Edit
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Toolmaxx-Agent
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Toolmaxx-ShellCommands
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Toolmaxx-Functions-Toolbench
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Toolmaxx-Functions-ToolACE
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Toolmaxx-Functions-apigen-subset
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-OpenAssistant2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-Opus-Merge-2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-sonnetorca-subset
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-sonnetorca-subset-2
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-Synthia
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-ASL
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-PersonaLLM-Opus
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-LongAlign
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-OpenLeecher-Instruct
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Assistantmaxx-Tulu3-IF
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Systemmaxx
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Logicmaxx-SAT-AP
type: dan-chat-advanced-llama3
- path: PJMixers/grimulkan_theory-of-mind-ShareGPT
type: dan-chat-advanced-llama3
- path: PJMixers/grimulkan_physical-reasoning-ShareGPT
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Reasoningmaxx-NaturalReasoning
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Reasoningmaxx-WebInstruct
type: dan-chat-advanced-llama3
- path: PocketDoc/Dans-Reasoningmaxx-GeneralReasoning
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-LN-V1-ShareGPT
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-Alpindale-LN-ShareGPT
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-Shoujo-AI-Filtered-ShareGPT
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-RP-Guild
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-OpenCAI-ShareGPT
type: dan-chat-advanced-llama3
- path: Delta-Vector/Orion-LIMARP-Complexity
type: dan-chat-advanced-llama3
shuffle_merged_datasets: true
dataset_prepared_path: /home/ubuntu/dpe-70b-data
val_set_size: 0.0
output_dir: /home/ubuntu/dpe-70b-fft-rev1
plugins:
- axolotl.integrations.liger.LigerPlugin
- axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: false
cut_cross_entropy: true
sequence_len: 32768
sample_packing: true
eval_sample_packing: true
pad_to_sequence_len: true
wandb_project: dpe-70b-fft
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:
gradient_accumulation_steps: 4
micro_batch_size: 1
num_epochs: 2
optimizer: paged_ademamix_8bit
optim_args: "beta1=0.9,beta2=0.999,beta3=0.999,alpha=5"
lr_scheduler: rex
learning_rate: 1.0e-6
max_grad_norm: 0.001
train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
s2_attention:
warmup_ratio: 0.1
evals_per_epoch:
eval_table_size:
eval_max_new_tokens:
saves_per_epoch: 2
debug:
deepspeed: ./deepspeed_configs/zero3_bf16.json
weight_decay: 0.0025
fsdp:
fsdp_config:
special_tokens:
eos_token: <|eot_id|>
pad_token: <|finetune_right_pad_id|>
home/ubuntu/dpe-70b-fft-rev1
This model is a fine-tuned version of meta-llama/Llama-3.1-70B on the merged.json, the PocketDoc/Dans-Prosemaxx-RP, the PocketDoc/Dans-Personamaxx-Logs-2, the PocketDoc/Dans-Personamaxx-VN, the PocketDoc/Dans-Kinomaxx-VanillaBackrooms, the PocketDoc/Dans-Prosemaxx-Gutenberg, the PocketDoc/Dans-Prosemaxx-Cowriter-3-XL, the PocketDoc/Dans-Prosemaxx-Adventure, the PocketDoc/Dans-Failuremaxx-Adventure-3, the PocketDoc/Dans-Prosemaxx-InstructWriter-ZeroShot-2, the PocketDoc/Dans-Prosemaxx-InstructWriter-ZeroShot-3, the PocketDoc/Dans-Prosemaxx-InstructWriter-Continue-2, the PocketDoc/Dans-Prosemaxx-Instructwriter-Long, the PocketDoc/Dans-Prosemaxx-RepRemover-1, the PocketDoc/Dans-MemoryCore-CoreCurriculum-Small, the AquaV/US-Army-Survival-Sharegpt, the AquaV/Multi-Environment-Operations-Sharegpt, the AquaV/Resistance-Sharegpt, the AquaV/Interrogation-Sharegpt, the AquaV/Chemical-Biological-Safety-Applications-Sharegpt, the AquaV/Energetic-Materials-Sharegpt, the PocketDoc/Dans-Mathmaxx, the PJMixers/Math-Multiturn-1K-ShareGPT, the PocketDoc/Dans-Taskmaxx, the PocketDoc/Dans-Taskmaxx-DataPrepper, the PocketDoc/Dans-Taskmaxx-ConcurrentQA-Reworked, the PocketDoc/Dans-Taskmaxx-TableGPT, the PocketDoc/Dans-Taskmaxx-SciRIFF, the PocketDoc/Dans-Taskmaxx-Edit, the PocketDoc/Dans-Toolmaxx-Agent, the PocketDoc/Dans-Toolmaxx-ShellCommands, the PocketDoc/Dans-Toolmaxx-Functions-Toolbench, the PocketDoc/Dans-Toolmaxx-Functions-ToolACE, the PocketDoc/Dans-Toolmaxx-Functions-apigen-subset, the PocketDoc/Dans-Assistantmaxx-OpenAssistant2, the PocketDoc/Dans-Assistantmaxx-Opus-Merge-2, the PocketDoc/Dans-Assistantmaxx-sonnetorca-subset, the PocketDoc/Dans-Assistantmaxx-sonnetorca-subset-2, the PocketDoc/Dans-Assistantmaxx-Synthia, the PocketDoc/Dans-Assistantmaxx-ASL, the PocketDoc/Dans-Assistantmaxx-PersonaLLM-Opus, the PocketDoc/Dans-Assistantmaxx-LongAlign, the PocketDoc/Dans-Assistantmaxx-OpenLeecher-Instruct, the PocketDoc/Dans-Assistantmaxx-Tulu3-IF, the PocketDoc/Dans-Systemmaxx, the PocketDoc/Dans-Logicmaxx-SAT-AP, the PJMixers/grimulkan_theory-of-mind-ShareGPT, the PJMixers/grimulkan_physical-reasoning-ShareGPT, the PocketDoc/Dans-Reasoningmaxx-NaturalReasoning, the PocketDoc/Dans-Reasoningmaxx-WebInstruct, the PocketDoc/Dans-Reasoningmaxx-GeneralReasoning, the Delta-Vector/Orion-LN-V1-ShareGPT, the Delta-Vector/Orion-Alpindale-LN-ShareGPT, the Delta-Vector/Orion-Shoujo-AI-Filtered-ShareGPT, the Delta-Vector/Orion-RP-Guild, the Delta-Vector/Orion-OpenCAI-ShareGPT and the Delta-Vector/Orion-LIMARP-Complexity datasets.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- total_eval_batch_size: 8
- optimizer: Use paged_ademamix_8bit and the args are: beta1=0.9,beta2=0.999,beta3=0.999,alpha=5
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 284
- training_steps: 2848
Training results
Framework versions
- Transformers 4.52.4
- Pytorch 2.7.1+cu128
- Datasets 3.6.0
- Tokenizers 0.21.2
Runs of NewEden DPE-70b-Ckpts on huggingface.co
8
Total runs
0
24-hour runs
-1
3-day runs
0
7-day runs
-1
30-day runs
More Information About DPE-70b-Ckpts huggingface.co Model
Url of DPE-70b-Ckpts
DPE-70b-Ckpts huggingface.co Url
Provider of DPE-70b-Ckpts huggingface.co
Other API from NewEden
Updated:September 20 2025
Total runs:
169
Run Growth:
163
Growth Rate:
96.45%
Total runs:
111
Run Growth:
71
Growth Rate:
63.96%
Total runs:
104
Run Growth:
0
Growth Rate:
0.00%
Total runs:
94
Run Growth:
0
Growth Rate:
0.00%
Total runs:
86
Run Growth:
0
Growth Rate:
0.00%
Total runs:
80
Run Growth:
0
Growth Rate:
0.00%
Total runs:
61
Run Growth:
0
Growth Rate:
0.00%
Total runs:
47
Run Growth:
47
Growth Rate:
100.00%
Total runs:
44
Run Growth:
0
Growth Rate:
0.00%
Total runs:
33
Run Growth:
27
Growth Rate:
81.82%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
27
Run Growth:
0
Growth Rate:
0.00%
Total runs:
23
Run Growth:
0
Growth Rate:
0.00%
Total runs:
21
Run Growth:
0
Growth Rate:
0.00%
Total runs:
21
Run Growth:
7
Growth Rate:
29.17%
Total runs:
15
Run Growth:
0
Growth Rate:
0.00%
Total runs:
15
Run Growth:
1
Growth Rate:
6.67%
Total runs:
15
Run Growth:
10
Growth Rate:
55.56%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-11
Growth Rate:
-100.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
4
Growth Rate:
40.00%
Total runs:
10
Run Growth:
10
Growth Rate:
100.00%
Total runs:
10
Run Growth:
-1
Growth Rate:
-9.09%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
4
Growth Rate:
44.44%
Total runs:
8
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
7
Growth Rate:
77.78%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
-164
Growth Rate:
-2342.86%