Total runs:
3.0K
Run Growth:
784
Growth Rate:
25.81%
Sponsored by
AdsCreator.com
nisten / shqiponja-15b-v1
Model's Last Updated: February 04 2024
Introduction of shqiponja-15b-v1
Model Details of shqiponja-15b-v1
15.6b 2expert MoE
See axolotl config
axolotl version:
0.4.0
base_model: nisten/shqiponja15
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
- path: iamshnoo/alpaca-cleaned-albanian
type: alpaca
shards: 10
- path: noxneural/lilium_albanicum_eng_alb
shards: 20
type:
field_system: system
field_instruction: question
field_output: response
format: "[INST] {instruction} [/INST]"
dataset_prepared_path: last_run_prepared
val_set_size: 0.0
output_dir: ./alora-out
# - model.layers.2[7-9]+.block_sparse_moe.experts.*
# - model.layers.3[0-9]+.block_sparse_moe.experts.*
# - model.layers.2[7-9]+.b
</details><br>
alora-out
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 10
- eval_batch_size: 10
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 80
- total_eval_batch_size: 40
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 3
Runs of nisten shqiponja-15b-v1 on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About shqiponja-15b-v1 huggingface.co Model
Url of shqiponja-15b-v1
shqiponja-15b-v1 huggingface.co Url
Provider of shqiponja-15b-v1 huggingface.co
Other API from nisten
Updated:August 07 2024
Total runs:
1.6K
Run Growth:
1.3K
Growth Rate:
78.52%
Total runs:
976
Run Growth:
-213
Growth Rate:
-21.82%
Total runs:
838
Run Growth:
60
Growth Rate:
6.77%
Total runs:
517
Run Growth:
278
Growth Rate:
57.44%
Total runs:
241
Run Growth:
-124
Growth Rate:
-51.45%
Total runs:
198
Run Growth:
134
Growth Rate:
67.68%
Total runs:
188
Run Growth:
-49
Growth Rate:
-26.06%
Total runs:
136
Run Growth:
136
Growth Rate:
100.00%
Total runs:
136
Run Growth:
-405
Growth Rate:
-297.79%
Total runs:
132
Run Growth:
4
Growth Rate:
3.03%
Total runs:
102
Run Growth:
18
Growth Rate:
17.65%
Total runs:
89
Run Growth:
43
Growth Rate:
48.31%
Total runs:
65
Run Growth:
0
Growth Rate:
0.00%
Total runs:
60
Run Growth:
-14
Growth Rate:
-20.90%
Total runs:
60
Run Growth:
-59
Growth Rate:
-96.72%
Total runs:
46
Run Growth:
-74
Growth Rate:
-160.87%
Total runs:
45
Run Growth:
0
Growth Rate:
0.00%
Total runs:
38
Run Growth:
4
Growth Rate:
10.53%
Total runs:
36
Run Growth:
3
Growth Rate:
8.33%
Total runs:
33
Run Growth:
-25
Growth Rate:
-73.53%
Total runs:
30
Run Growth:
18
Growth Rate:
60.00%
Total runs:
29
Run Growth:
12
Growth Rate:
44.44%
Total runs:
26
Run Growth:
0
Growth Rate:
0.00%
Total runs:
24
Run Growth:
10
Growth Rate:
41.67%
Total runs:
17
Run Growth:
5
Growth Rate:
29.41%
Total runs:
16
Run Growth:
-9
Growth Rate:
-56.25%
Total runs:
13
Run Growth:
3
Growth Rate:
23.08%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-19
Growth Rate:
-172.73%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
9
Run Growth:
5
Growth Rate:
55.56%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
6
Run Growth:
-8
Growth Rate:
-133.33%
Total runs:
6
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
-1
Growth Rate:
-20.00%
Total runs:
5
Run Growth:
-27
Growth Rate:
-540.00%
Total runs:
3
Run Growth:
-1
Growth Rate:
-33.33%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%