Total runs:
1.0K
Run Growth:
161
Growth Rate:
15.89%
Sponsored by
AdsCreator.com
nbeerbower / bophades-mistral-math-DPO-7B
Model's Last Updated: April 06 2024
text-generation
Introduction of bophades-mistral-math-DPO-7B
Model Details of bophades-mistral-math-DPO-7B
bophades-mistral-math-DPO-7B
bophades-v2-mistral-7B finetuned on kyujinpy/orca_math_dpo .
Method
Finetuned using an A100 on Google Colab. 🙏
Fine-tune a Mistral-7b model with Direct Preference Optimization - Maxime Labonne
Configuration
LoRA, model, and training settings:
# LoRA configuration
peft_config = LoraConfig(
r=16,
lora_alpha=16,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']
)
# Model to fine-tune
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
load_in_4bit=True
)
model.config.use_cache = False
# Reference model
ref_model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
load_in_4bit=True
)
# Training arguments
training_args = TrainingArguments(
per_device_train_batch_size=2,
gradient_accumulation_steps=2,
gradient_checkpointing=True,
learning_rate=2e-5,
lr_scheduler_type="cosine",
max_steps=420,
save_strategy="no",
logging_steps=1,
output_dir=new_model,
optim="paged_adamw_32bit",
warmup_steps=100,
bf16=True,
report_to="wandb",
)
# Create DPO trainer
dpo_trainer = DPOTrainer(
model,
ref_model,
args=training_args,
train_dataset=dataset,
tokenizer=tokenizer,
peft_config=peft_config,
beta=0.1,
max_prompt_length=1024,
max_length=1536,
force_use_ref_model=True
)
# Fine-tune model with DPO
dpo_trainer.train()
Runs of nbeerbower bophades-mistral-math-DPO-7B on huggingface.co
27
Total runs
-4
24-hour runs
-3
3-day runs
1
7-day runs
7
30-day runs
More Information About bophades-mistral-math-DPO-7B huggingface.co Model
Url of bophades-mistral-math-DPO-7B
bophades-mistral-math-DPO-7B huggingface.co Url
Provider of bophades-mistral-math-DPO-7B huggingface.co
Other API from nbeerbower
Updated:June 10 2025
Total runs:
428
Run Growth:
323
Growth Rate:
75.47%
Total runs:
270
Run Growth:
270
Growth Rate:
100.00%
Total runs:
172
Run Growth:
63
Growth Rate:
36.63%
Total runs:
147
Run Growth:
-16
Growth Rate:
-10.88%
Total runs:
135
Run Growth:
-42
Growth Rate:
-31.11%
Total runs:
115
Run Growth:
-36
Growth Rate:
-31.30%
Total runs:
111
Run Growth:
-307
Growth Rate:
-276.58%
Total runs:
110
Run Growth:
81
Growth Rate:
73.64%
Total runs:
94
Run Growth:
22
Growth Rate:
23.40%
Total runs:
93
Run Growth:
51
Growth Rate:
54.84%
Total runs:
93
Run Growth:
6
Growth Rate:
6.45%
Total runs:
85
Run Growth:
0
Growth Rate:
0.00%
Total runs:
83
Run Growth:
-3
Growth Rate:
-3.61%
Total runs:
76
Run Growth:
41
Growth Rate:
53.95%
Total runs:
73
Run Growth:
32
Growth Rate:
43.84%
Total runs:
73
Run Growth:
35
Growth Rate:
47.95%
Total runs:
73
Run Growth:
44
Growth Rate:
60.27%
Total runs:
67
Run Growth:
-109
Growth Rate:
-162.69%
Total runs:
67
Run Growth:
-20
Growth Rate:
-29.41%
Total runs:
66
Run Growth:
18
Growth Rate:
27.27%
Total runs:
64
Run Growth:
31
Growth Rate:
48.44%
Total runs:
63
Run Growth:
4
Growth Rate:
6.35%
Total runs:
62
Run Growth:
-182
Growth Rate:
-293.55%
Total runs:
61
Run Growth:
-27
Growth Rate:
-44.26%
Total runs:
61
Run Growth:
43
Growth Rate:
70.49%
Total runs:
59
Run Growth:
-5
Growth Rate:
-8.47%
Total runs:
59
Run Growth:
14
Growth Rate:
23.73%
Total runs:
58
Run Growth:
52
Growth Rate:
89.66%
Total runs:
58
Run Growth:
26
Growth Rate:
44.83%
Total runs:
58
Run Growth:
-10
Growth Rate:
-17.24%
Total runs:
57
Run Growth:
29
Growth Rate:
50.88%
Total runs:
55
Run Growth:
-8
Growth Rate:
-14.55%
Total runs:
55
Run Growth:
14
Growth Rate:
25.45%
Total runs:
54
Run Growth:
-26
Growth Rate:
-48.15%
Total runs:
52
Run Growth:
21
Growth Rate:
40.38%
Total runs:
51
Run Growth:
15
Growth Rate:
29.41%
Total runs:
51
Run Growth:
27
Growth Rate:
52.94%
Total runs:
50
Run Growth:
33
Growth Rate:
66.00%
Total runs:
50
Run Growth:
16
Growth Rate:
32.00%
Total runs:
49
Run Growth:
47
Growth Rate:
95.92%
Total runs:
49
Run Growth:
10
Growth Rate:
20.41%
Total runs:
49
Run Growth:
10
Growth Rate:
20.41%
Total runs:
49
Run Growth:
29
Growth Rate:
59.18%
Total runs:
48
Run Growth:
35
Growth Rate:
70.00%
Total runs:
48
Run Growth:
28
Growth Rate:
58.33%
Total runs:
48
Run Growth:
18
Growth Rate:
37.50%
Total runs:
47
Run Growth:
16
Growth Rate:
34.04%
Total runs:
47
Run Growth:
23
Growth Rate:
48.94%
Total runs:
47
Run Growth:
-4
Growth Rate:
-8.51%
Total runs:
47
Run Growth:
27
Growth Rate:
57.45%
Total runs:
47
Run Growth:
27
Growth Rate:
57.45%
Total runs:
47
Run Growth:
47
Growth Rate:
100.00%
Total runs:
46
Run Growth:
23
Growth Rate:
50.00%
Total runs:
46
Run Growth:
39
Growth Rate:
84.78%
Total runs:
46
Run Growth:
22
Growth Rate:
47.83%
Total runs:
44
Run Growth:
5
Growth Rate:
11.36%
Total runs:
44
Run Growth:
3
Growth Rate:
6.82%
Total runs:
44
Run Growth:
26
Growth Rate:
59.09%
Total runs:
43
Run Growth:
10
Growth Rate:
23.26%
Total runs:
43
Run Growth:
-4
Growth Rate:
-9.30%
Total runs:
42
Run Growth:
31
Growth Rate:
73.81%
Total runs:
40
Run Growth:
21
Growth Rate:
52.50%
Total runs:
40
Run Growth:
20
Growth Rate:
50.00%
Total runs:
39
Run Growth:
13
Growth Rate:
33.33%
Total runs:
38
Run Growth:
-2
Growth Rate:
-5.26%
Total runs:
38
Run Growth:
20
Growth Rate:
52.63%
Total runs:
38
Run Growth:
5
Growth Rate:
13.16%
Total runs:
37
Run Growth:
28
Growth Rate:
75.68%
Total runs:
36
Run Growth:
21
Growth Rate:
56.76%
Total runs:
37
Run Growth:
25
Growth Rate:
67.57%
Total runs:
36
Run Growth:
21
Growth Rate:
58.33%
Total runs:
36
Run Growth:
6
Growth Rate:
16.67%
Total runs:
36
Run Growth:
23
Growth Rate:
63.89%
Total runs:
35
Run Growth:
24
Growth Rate:
68.57%
Total runs:
35
Run Growth:
23
Growth Rate:
65.71%
Total runs:
35
Run Growth:
13
Growth Rate:
36.11%
Total runs:
35
Run Growth:
0
Growth Rate:
0.00%
Total runs:
34
Run Growth:
-21
Growth Rate:
-60.00%
Total runs:
34
Run Growth:
19
Growth Rate:
55.88%
Total runs:
34
Run Growth:
16
Growth Rate:
47.06%
Total runs:
33
Run Growth:
20
Growth Rate:
60.61%
Total runs:
33
Run Growth:
13
Growth Rate:
39.39%
Total runs:
33
Run Growth:
-8
Growth Rate:
-25.00%
Total runs:
32
Run Growth:
10
Growth Rate:
31.25%