jeiku / NeuroNoise_v1

huggingface.co
Total runs: 7
24-hour runs: 0
7-day runs: -1
30-day runs: 3
Model's Last Updated: August 31 2024
text-generation

Introduction of NeuroNoise_v1

Model Details of NeuroNoise_v1

Built with Axolotl

See axolotl config

axolotl version: 0.4.1

base_model: IntervitensInc/Llama-3.1-Minitron-4B-Width-Base-chatml
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: NewEden/Gryphe-3.5-16k-Subset
    type: sharegpt
    conversation: chatml
  - path: ResplendentAI/bluemoon
    type: sharegpt
    conversation: chatml
  - path: openerotica/freedom-rp
    type: sharegpt
    conversation: chatml
  - path: MinervaAI/Aesir-Preview
    type: sharegpt
    conversation: chatml
  - path: anthracite-org/stheno-filtered-v1.1
    type: sharegpt
    conversation: chatml
  - path: NewEden/Kalo-Opus-Instruct-22k-Refusal-Murdered
    type: sharegpt
    conversation: chatml
  - path: Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
    type: sharegpt
    conversation: chatml
  - path: Epiculous/SynthRP-Gens-v1.1-Filtered-n-Cleaned
    type: sharegpt
    conversation: chatml
  - path: jeiku/jeikutxt
    type: completion
  - path: ResplendentAI/Sissification_Hypno_1k
    type: alpaca
  - path: ResplendentAI/theory_of_mind_fixed_output
    type: alpaca
  - path: ResplendentAI/Synthetic_Soul_1k
    type: alpaca

chat_template: chatml

val_set_size: 0.01
output_dir: ./outputs/out

adapter:
lora_r:
lora_alpha:
lora_dropout:
lora_target_linear:

sequence_len: 8192
# sequence_len: 32768
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: true

plugins:
  - axolotl.integrations.liger.LigerPlugin
liger_rope: true
liger_rms_norm: true
liger_swiglu: true
liger_fused_linear_cross_entropy: true

wandb_project: Neuro4B
wandb_entity:
wandb_watch:
wandb_name: Neuro4B
wandb_log_model:

gradient_accumulation_steps: 32
micro_batch_size: 2
num_epochs: 2
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.00001
weight_decay: 0.05

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: true

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_ratio: 0.1
evals_per_epoch: 4
eval_table_size:
eval_max_new_tokens: 128
saves_per_epoch: 2

debug:
deepspeed: deepspeed_configs/zero3.json
fsdp:
fsdp_config:

special_tokens:
  pad_token: <|finetune_right_pad_id|>


outputs/out

This model is a fine-tuned version of IntervitensInc/Llama-3.1-Minitron-4B-Width-Base-chatml on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.4207
Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure
Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • gradient_accumulation_steps: 32
  • total_train_batch_size: 128
  • total_eval_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 19
  • num_epochs: 2
Training results
Training Loss Epoch Step Validation Loss
2.0722 0.0094 1 3.0240
1.5947 0.2529 27 2.8578
1.4579 0.5057 54 2.5541
1.4422 0.7586 81 2.4550
1.4039 1.0006 108 2.4334
1.3428 1.2534 135 2.4217
1.3054 1.5063 162 2.4259
1.3378 1.7591 189 2.4207
Framework versions
  • Transformers 4.45.0.dev0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1

Runs of jeiku NeuroNoise_v1 on huggingface.co

7
Total runs
0
24-hour runs
0
3-day runs
-1
7-day runs
3
30-day runs

More Information About NeuroNoise_v1 huggingface.co Model

More NeuroNoise_v1 license Visit here:

https://choosealicense.com/licenses/other

NeuroNoise_v1 huggingface.co

NeuroNoise_v1 huggingface.co is an AI model on huggingface.co that provides NeuroNoise_v1's model effect (), which can be used instantly with this jeiku NeuroNoise_v1 model. huggingface.co supports a free trial of the NeuroNoise_v1 model, and also provides paid use of the NeuroNoise_v1. Support call NeuroNoise_v1 model through api, including Node.js, Python, http.

NeuroNoise_v1 huggingface.co Url

https://huggingface.co/jeiku/NeuroNoise_v1

jeiku NeuroNoise_v1 online free

NeuroNoise_v1 huggingface.co is an online trial and call api platform, which integrates NeuroNoise_v1's modeling effects, including api services, and provides a free online trial of NeuroNoise_v1, you can try NeuroNoise_v1 online for free by clicking the link below.

jeiku NeuroNoise_v1 online free url in huggingface.co:

https://huggingface.co/jeiku/NeuroNoise_v1

NeuroNoise_v1 install

NeuroNoise_v1 is an open source model from GitHub that offers a free installation service, and any user can find NeuroNoise_v1 on GitHub to install. At the same time, huggingface.co provides the effect of NeuroNoise_v1 install, users can directly use NeuroNoise_v1 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

NeuroNoise_v1 install url in huggingface.co:

https://huggingface.co/jeiku/NeuroNoise_v1

Url of NeuroNoise_v1

NeuroNoise_v1 huggingface.co Url

Provider of NeuroNoise_v1 huggingface.co

jeiku
ORGANIZATIONS

Other API from jeiku

huggingface.co

Total runs: 143
Run Growth: 113
Growth Rate: 79.02%
Updated:February 29 2024
huggingface.co

Total runs: 124
Run Growth: 3
Growth Rate: 2.42%
Updated:April 02 2024
huggingface.co

Total runs: 93
Run Growth: -12
Growth Rate: -12.90%
Updated:February 19 2024
huggingface.co

Total runs: 92
Run Growth: -13
Growth Rate: -14.13%
Updated:March 01 2024
huggingface.co

Total runs: 92
Run Growth: -19
Growth Rate: -20.65%
Updated:February 22 2024
huggingface.co

Total runs: 92
Run Growth: -19
Growth Rate: -20.65%
Updated:February 17 2024
huggingface.co

Total runs: 90
Run Growth: -23
Growth Rate: -25.56%
Updated:March 19 2024
huggingface.co

Total runs: 72
Run Growth: 55
Growth Rate: 76.39%
Updated:January 17 2024
huggingface.co

Total runs: 47
Run Growth: -29
Growth Rate: -61.70%
Updated:January 21 2024
huggingface.co

Total runs: 42
Run Growth: 0
Growth Rate: 0.00%
Updated:December 06 2024
huggingface.co

Total runs: 41
Run Growth: -46
Growth Rate: -112.20%
Updated:January 21 2024
huggingface.co

Total runs: 28
Run Growth: -16
Growth Rate: -57.14%
Updated:February 22 2024
huggingface.co

Total runs: 25
Run Growth: -10
Growth Rate: -40.00%
Updated:February 03 2024
huggingface.co

Total runs: 18
Run Growth: 0
Growth Rate: 0.00%
Updated:February 13 2024
huggingface.co

Total runs: 18
Run Growth: -11
Growth Rate: -61.11%
Updated:February 13 2024
huggingface.co

Total runs: 18
Run Growth: 0
Growth Rate: 0.00%
Updated:September 27 2024
huggingface.co

Total runs: 18
Run Growth: -30
Growth Rate: -166.67%
Updated:January 23 2024
huggingface.co

Total runs: 15
Run Growth: 11
Growth Rate: 73.33%
Updated:December 27 2023
huggingface.co

Total runs: 14
Run Growth: 11
Growth Rate: 78.57%
Updated:March 18 2024
huggingface.co

Total runs: 12
Run Growth: 11
Growth Rate: 91.67%
Updated:October 09 2024
huggingface.co

Total runs: 11
Run Growth: 6
Growth Rate: 54.55%
Updated:February 13 2024
huggingface.co

Total runs: 11
Run Growth: 3
Growth Rate: 27.27%
Updated:February 02 2024
huggingface.co

Total runs: 11
Run Growth: 1
Growth Rate: 9.09%
Updated:July 02 2024
huggingface.co

Total runs: 11
Run Growth: 0
Growth Rate: 0.00%
Updated:September 05 2024
huggingface.co

Total runs: 11
Run Growth: -8
Growth Rate: -72.73%
Updated:December 27 2023
huggingface.co

Total runs: 9
Run Growth: 6
Growth Rate: 66.67%
Updated:August 31 2024
huggingface.co

Total runs: 9
Run Growth: 1
Growth Rate: 11.11%
Updated:August 18 2024
huggingface.co

Total runs: 9
Run Growth: 2
Growth Rate: 22.22%
Updated:January 23 2024
huggingface.co

Total runs: 8
Run Growth: 3
Growth Rate: 37.50%
Updated:February 14 2024
huggingface.co

Total runs: 8
Run Growth: 3
Growth Rate: 37.50%
Updated:June 14 2024
huggingface.co

Total runs: 8
Run Growth: 3
Growth Rate: 37.50%
Updated:January 22 2024
huggingface.co

Total runs: 8
Run Growth: -30
Growth Rate: -375.00%
Updated:January 21 2024
huggingface.co

Total runs: 8
Run Growth: 6
Growth Rate: 75.00%
Updated:September 22 2024
huggingface.co

Total runs: 8
Run Growth: 3
Growth Rate: 37.50%
Updated:March 05 2024
huggingface.co

Total runs: 8
Run Growth: 6
Growth Rate: 75.00%
Updated:December 04 2024
huggingface.co

Total runs: 8
Run Growth: 4
Growth Rate: 50.00%
Updated:May 22 2024
huggingface.co

Total runs: 7
Run Growth: 1
Growth Rate: 14.29%
Updated:January 20 2024
huggingface.co

Total runs: 7
Run Growth: -3
Growth Rate: -42.86%
Updated:January 08 2024
huggingface.co

Total runs: 7
Run Growth: 4
Growth Rate: 57.14%
Updated:October 09 2024
huggingface.co

Total runs: 7
Run Growth: -28
Growth Rate: -400.00%
Updated:December 27 2023
huggingface.co

Total runs: 7
Run Growth: 2
Growth Rate: 28.57%
Updated:February 13 2024
huggingface.co

Total runs: 7
Run Growth: 3
Growth Rate: 42.86%
Updated:March 18 2024
huggingface.co

Total runs: 7
Run Growth: 3
Growth Rate: 42.86%
Updated:June 13 2024
huggingface.co

Total runs: 7
Run Growth: 0
Growth Rate: 0.00%
Updated:September 22 2024
huggingface.co

Total runs: 7
Run Growth: -1
Growth Rate: -14.29%
Updated:January 22 2024
huggingface.co

Total runs: 7
Run Growth: -5
Growth Rate: -71.43%
Updated:December 29 2023