Sponsored by
APIMart
Azurro / llama-2-7b-qlora-polish
Model's Last Updated: September 25 2023
Introduction of llama-2-7b-qlora-polish
Model Details of llama-2-7b-qlora-polish
This repo contains a qlora adapter for Llama-2-7b , trained on 1B tokens, only in Polish.
The training took 20 days on a single RTX 4090 with the following hyperparameters:
- context length: 4096
- batch_size: 128
- learning_rate: 0.0002, cosine with warmup
- lora_r: 64
- lora_alpha: 16
- lora_modules: all
- lora_dropout: 0.0
- weight_decay: 0.1
- max_grad_norm: 0.3
- double_quant, nf4
- optimizer: paged_adamw_32bit
This adapter allows the model to speak Polish more accurately than vanilla Llama-2-7b .
Runs of Azurro llama-2-7b-qlora-polish on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About llama-2-7b-qlora-polish huggingface.co Model
Url of llama-2-7b-qlora-polish
llama-2-7b-qlora-polish huggingface.co Url
Provider of llama-2-7b-qlora-polish huggingface.co
Other API from Azurro
Total runs:
3.4K
Run Growth:
1.2K
Growth Rate:
35.66%
Total runs:
112
Run Growth:
93
Growth Rate:
83.04%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%