Total runs:
680
Run Growth:
566
Growth Rate:
79.94%
Sponsored by
APIMart
AiAF / rp-2b
Model's Last Updated: March 30 2026
text-generation
Introduction of rp-2b
Model Details of rp-2b
See axolotl config
axolotl version:
0.13.0.dev0
# 1. Base Model & Tokenizer
base_model: google/gemma-2-2b-it
model_type: AutoModelForCausalLM # Corrected from 'type_of_model' for axolotl
tokenizer_type: AutoTokenizer
hub_model_id: AiAF/rp-2b # New model ID for this finetune
hub_strategy: checkpoint
# 2. LoRA / QLoRA Configuration
load_in_4bit: true
adapter: qlora
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_linear: true
# 3. Dataset Configuration (TRAIN = streamed)
streaming: true
streaming_multipack_buffer_size: 10000
sample_packing: true
datasets:
- path: AiAF/conversations
data_files: conversations_V3.jsonl
# revision:
type: chat_template
split: train
field_messages: conversations
message_property_mappings:
role: from
content: value
chat_template: jinja
chat_template_jinja: |
{{ bos_token }}
{% for m in messages %}
{% set role = 'model' if m['role']=='assistant' else 'user' %}
{{ '<start_of_turn>' + role + '\n' + m['content'] | trim + '<end_of_turn>\n' }}
{% endfor %}
{% if add_generation_prompt %}
{{ '<start_of_turn>model\n' }}
{% endif %}
# chat_template_jinja: |
# {{ bos_token }}
# {% set last = None %}
# {% for m in messages %}
# {% set raw_role = 'model' if m['role']=='assistant' else m['role'] %}
# {% set role = 'user' if raw_role=='system' else raw_role %}
# {% if role == last and role == 'user' %}
# {{ m['content'] | trim }}
# {% else %}
# {{ '<start_of_turn>' + role + '\n' + m['content'] | trim + '<end_of_turn>\n' }}
# {% endif %}
# {% set last = role %}
# {% endfor %}
# {% if add_generation_prompt %}
# {{ '<start_of_turn>model\n' }}
# {% endif %}
roles_to_train: ["assistant"]
train_on_eos: "turn"
# Use a fixed (non-streamed) eval file with the same schema/Jinja
test_datasets:
- path: .
name: json
type: chat_template
data_files: eval-datasets/shuf-1000_conversations_V2.jsonl # small, representative eval slice
split: train
field_messages: conversations
message_property_mappings:
role: from
content: value
chat_template: jinja
chat_template_jinja: |
{{ bos_token }}
{% for m in messages %}
{% set role = 'model' if m['role']=='assistant' else 'user' %}
{{ '<start_of_turn>' + role + '\n' + m['content'] | trim + '<end_of_turn>\n' }}
{% endfor %}
{% if add_generation_prompt %}
{{ '<start_of_turn>model\n' }}
{% endif %}
# chat_template_jinja: |
# {{ bos_token }}
# {% set last = None %}
# {% for m in messages %}
# {% set raw_role = 'model' if m['role']=='assistant' else m['role'] %}
# {% set role = 'user' if raw_role=='system' else raw_role %}
# {% if role == last and role == 'user' %}
# {{ m['content'] | trim }}
# {% else %}
# {{ '<start_of_turn>' + role + '\n' + m['content'] | trim + '<end_of_turn>\n' }}
# {% endif %}
# {% set last = role %}
# {% endfor %}
# {% if add_generation_prompt %}
# {{ '<start_of_turn>model\n' }}
# {% endif %}
roles_to_train: ["assistant"]
# 4. Training Parameters
sequence_len: 2048
sample_packing: true
eval_sample_packing: true
# val_set_size: 0.05 # remove for streaming
# num_epochs: 10 # replace epochs with max_steps
max_steps: 1000 # set your target steps
dataset_prepared_path: last_run_prepared
# 5. Saving and Evaluation Strategy (use steps with streaming)
evaluation_strategy: steps
save_strategy: steps
eval_steps: 50
save_steps: 50
save_total_limit: 100
resume_from_checkpoint:
# 6. Output & Logging
output_dir: ./outputs/sft/gemma-2-2b-it-rp-sft-qlora
wandb_project: "rp-sft"
wandb_name: "gemma-2-2b-it-rp-sft-qlora"
wandb_log_model: "false"
wandb_run_id: "gemma-2-2b-it-rp-sft-qlora"
# 7. Batching & Optimizer
gradient_accumulation_steps: 4
micro_batch_size: 2
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002
weight_decay: 0.0
# 8. Hardware & Performance
bf16: true
#fp16: true
tf32: true
flash_attention: true
gradient_checkpointing: true
logging_steps: 1
# 9. Special Tokens
eot_tokens: ["<end_of_turn>"]
special_tokens:
bos_token: "<bos>"
eos_token: "<eos>"
pad_token: "<pad>"
rp-2b
This model is a fine-tuned version of google/gemma-2-2b-it on the AiAF/conversations dataset. It achieves the following results on the evaluation set:
- Loss: 2.2455
- Memory/max Active (gib): 7.78
- Memory/max Allocated (gib): 7.78
- Memory/device Reserved (gib): 17.79
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 4
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 30
- training_steps: 1000
Training results
| Training Loss | Epoch | Step | Validation Loss | Active (gib) | Allocated (gib) | Reserved (gib) |
|---|---|---|---|---|---|---|
| No log | 0 | 0 | 3.1654 | 7.61 | 7.61 | 8.66 |
| 2.7377 | 0.05 | 50 | 2.5978 | 7.78 | 7.78 | 17.75 |
| 2.3997 | 0.1 | 100 | 2.5592 | 7.78 | 7.78 | 17.79 |
| 2.6275 | 0.15 | 150 | 2.5410 | 7.78 | 7.78 | 17.79 |
| 2.8182 | 0.2 | 200 | 2.5224 | 7.78 | 7.78 | 17.79 |
| 2.4428 | 0.25 | 250 | 2.4962 | 7.78 | 7.78 | 17.79 |
| 2.6206 | 0.3 | 300 | 2.4672 | 7.78 | 7.78 | 17.79 |
| 2.4492 | 0.35 | 350 | 2.4435 | 7.78 | 7.78 | 17.79 |
| 2.2787 | 0.4 | 400 | 2.4185 | 7.78 | 7.78 | 17.79 |
| 2.541 | 0.45 | 450 | 2.3998 | 7.78 | 7.78 | 17.79 |
| 2.5542 | 0.5 | 500 | 2.3640 | 7.78 | 7.78 | 17.79 |
| 2.6825 | 0.55 | 550 | 2.3484 | 7.78 | 7.78 | 17.79 |
| 2.6304 | 0.6 | 600 | 2.3278 | 7.78 | 7.78 | 17.79 |
| 2.4854 | 0.65 | 650 | 2.3104 | 7.78 | 7.78 | 17.79 |
| 2.3788 | 0.7 | 700 | 2.2877 | 7.78 | 7.78 | 17.79 |
| 2.2126 | 0.75 | 750 | 2.2748 | 7.78 | 7.78 | 17.79 |
| 2.4695 | 0.8 | 800 | 2.2662 | 7.78 | 7.78 | 17.79 |
| 2.5086 | 0.85 | 850 | 2.2553 | 7.78 | 7.78 | 17.79 |
| 2.404 | 0.9 | 900 | 2.2489 | 7.78 | 7.78 | 17.79 |
| 2.4012 | 0.95 | 950 | 2.2460 | 7.78 | 7.78 | 17.79 |
| 2.2586 | 1.0 | 1000 | 2.2455 | 7.78 | 7.78 | 17.79 |
Framework versions
- PEFT 0.17.1
- Transformers 4.57.0
- Pytorch 2.7.1+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1
Runs of AiAF rp-2b on huggingface.co
11
Total runs
0
24-hour runs
0
3-day runs
2
7-day runs
5
30-day runs
More Information About rp-2b huggingface.co Model
Url of rp-2b
rp-2b huggingface.co Url
Provider of rp-2b huggingface.co
Other API from AiAF
Updated:January 10 2025
Total runs:
258
Run Growth:
246
Growth Rate:
95.72%
Total runs:
250
Run Growth:
103
Growth Rate:
42.39%
Total runs:
230
Run Growth:
107
Growth Rate:
45.92%
Total runs:
187
Run Growth:
-101
Growth Rate:
-55.49%
Total runs:
157
Run Growth:
121
Growth Rate:
75.16%
Total runs:
152
Run Growth:
-9
Growth Rate:
-6.08%
Total runs:
132
Run Growth:
-220
Growth Rate:
-165.41%
Total runs:
130
Run Growth:
125
Growth Rate:
91.91%
Total runs:
128
Run Growth:
101
Growth Rate:
76.52%
Total runs:
113
Run Growth:
96
Growth Rate:
86.49%
Total runs:
111
Run Growth:
82
Growth Rate:
72.57%
Total runs:
103
Run Growth:
-196
Growth Rate:
-194.06%
Total runs:
100
Run Growth:
63
Growth Rate:
62.38%
Total runs:
90
Run Growth:
52
Growth Rate:
61.18%
Total runs:
79
Run Growth:
81
Growth Rate:
75.70%
Total runs:
73
Run Growth:
64
Growth Rate:
83.12%
Total runs:
63
Run Growth:
-179
Growth Rate:
-271.21%
Total runs:
58
Run Growth:
-7
Growth Rate:
-11.86%
Total runs:
51
Run Growth:
21
Growth Rate:
38.89%
Total runs:
50
Run Growth:
40
Growth Rate:
81.63%
Total runs:
49
Run Growth:
17
Growth Rate:
34.00%
Total runs:
49
Run Growth:
33
Growth Rate:
64.71%
Total runs:
47
Run Growth:
39
Growth Rate:
81.25%
Total runs:
46
Run Growth:
10
Growth Rate:
20.83%
Total runs:
46
Run Growth:
46
Growth Rate:
100.00%
Total runs:
45
Run Growth:
24
Growth Rate:
53.33%
Total runs:
40
Run Growth:
26
Growth Rate:
65.00%
Total runs:
38
Run Growth:
2
Growth Rate:
5.13%
Total runs:
35
Run Growth:
-7
Growth Rate:
-18.92%
Total runs:
35
Run Growth:
22
Growth Rate:
62.86%
Total runs:
34
Run Growth:
33
Growth Rate:
100.00%
Total runs:
32
Run Growth:
21
Growth Rate:
65.63%
Total runs:
32
Run Growth:
-48
Growth Rate:
-145.45%
Total runs:
31
Run Growth:
-5
Growth Rate:
-15.15%
Total runs:
30
Run Growth:
-2
Growth Rate:
-6.45%
Total runs:
29
Run Growth:
10
Growth Rate:
31.25%
Total runs:
28
Run Growth:
-8
Growth Rate:
-27.59%
Total runs:
26
Run Growth:
-9
Growth Rate:
-34.62%
Total runs:
24
Run Growth:
3
Growth Rate:
13.04%
Total runs:
24
Run Growth:
8
Growth Rate:
33.33%
Total runs:
24
Run Growth:
-28
Growth Rate:
-121.74%
Total runs:
23
Run Growth:
16
Growth Rate:
69.57%
Total runs:
23
Run Growth:
0
Growth Rate:
0.00%
Total runs:
23
Run Growth:
18
Growth Rate:
75.00%
Total runs:
23
Run Growth:
14
Growth Rate:
60.87%
Total runs:
22
Run Growth:
13
Growth Rate:
59.09%
Total runs:
22
Run Growth:
6
Growth Rate:
27.27%
Total runs:
22
Run Growth:
-9
Growth Rate:
-37.50%
Total runs:
21
Run Growth:
15
Growth Rate:
71.43%
Total runs:
21
Run Growth:
-17
Growth Rate:
-94.44%
Total runs:
21
Run Growth:
5
Growth Rate:
21.74%
Total runs:
21
Run Growth:
-4
Growth Rate:
-19.05%
Total runs:
20
Run Growth:
2
Growth Rate:
10.00%
Total runs:
20
Run Growth:
-18
Growth Rate:
-85.71%
Total runs:
20
Run Growth:
9
Growth Rate:
45.00%
Total runs:
20
Run Growth:
-31
Growth Rate:
-163.16%
Total runs:
20
Run Growth:
8
Growth Rate:
42.11%
Total runs:
19
Run Growth:
16
Growth Rate:
59.26%
Total runs:
19
Run Growth:
2
Growth Rate:
10.53%
Total runs:
18
Run Growth:
10
Growth Rate:
58.82%
Total runs:
17
Run Growth:
5
Growth Rate:
27.78%
Total runs:
17
Run Growth:
15
Growth Rate:
93.75%
Total runs:
17
Run Growth:
10
Growth Rate:
58.82%
Total runs:
16
Run Growth:
-7
Growth Rate:
-46.67%
Total runs:
16
Run Growth:
-11
Growth Rate:
-73.33%
Total runs:
16
Run Growth:
10
Growth Rate:
66.67%
Total runs:
15
Run Growth:
-13
Growth Rate:
-86.67%
Total runs:
14
Run Growth:
-6
Growth Rate:
-37.50%
Total runs:
14
Run Growth:
7
Growth Rate:
36.84%
Total runs:
14
Run Growth:
4
Growth Rate:
25.00%
Total runs:
14
Run Growth:
4
Growth Rate:
28.57%
Total runs:
13
Run Growth:
3
Growth Rate:
25.00%
Total runs:
13
Run Growth:
8
Growth Rate:
61.54%
Total runs:
12
Run Growth:
4
Growth Rate:
36.36%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-3
Growth Rate:
-27.27%
Total runs:
10
Run Growth:
2
Growth Rate:
22.22%
Total runs:
10
Run Growth:
7
Growth Rate:
77.78%
Total runs:
10
Run Growth:
10
Growth Rate:
100.00%
Total runs:
10
Run Growth:
5
Growth Rate:
45.45%