Total runs:
357.6K
Run Growth:
-69.4K
Growth Rate:
-19.40%
Sponsored by
iMean AI Trip Planner
mlabonne / OrpoLlama-3-8B
Model's Last Updated: June 15 2024
text-generation
Introduction of OrpoLlama-3-8B
Model Details of OrpoLlama-3-8B
OrpoLlama-3-8B
This is an ORPO fine-tune of meta-llama/Meta-Llama-3-8B on mlabonne/orpo-dpo-mix-40k created for this article .
It's a successful fine-tune that follows the ChatML template!
Try the demo : https://huggingface.co/spaces/mlabonne/OrpoLlama-3-8B
🔎 Application
This model uses a context window of 8k. It was trained with the ChatML template.
⚡ Quantized models
Thanks to bartowski, solidrust, and LoneStriker for the quantized models.
- GGUF : https://huggingface.co/bartowski/OrpoLlama-3-8B-GGUF
- AWQ : https://huggingface.co/solidrust/OrpoLlama-3-8B-AWQ
-
EXL2
:
- https://huggingface.co/LoneStriker/OrpoLlama-3-8B-3.0bpw-h6-exl2
- https://huggingface.co/LoneStriker/OrpoLlama-3-8B-4.0bpw-h6-exl2
- https://huggingface.co/LoneStriker/OrpoLlama-3-8B-5.0bpw-h6-exl2
- https://huggingface.co/LoneStriker/OrpoLlama-3-8B-6.0bpw-h6-exl2
- https://huggingface.co/LoneStriker/OrpoLlama-3-8B-8.0bpw-h8-exl2
🏆 Evaluation
Nous
OrpoLlama-4-8B outperforms Llama-3-8B-Instruct on the GPT4All and TruthfulQA datasets.
Evaluation performed using LLM AutoEval , see the entire leaderboard here .
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|---|---|---|---|---|---|
| meta-llama/Meta-Llama-3-8B-Instruct 📄 | 51.34 | 41.22 | 69.86 | 51.65 | 42.64 |
| mlabonne/OrpoLlama-3-8B 📄 | 48.63 | 34.17 | 70.59 | 52.39 | 37.36 |
| mlabonne/OrpoLlama-3-8B-1k 📄 | 46.76 | 31.56 | 70.19 | 48.11 | 37.17 |
| meta-llama/Meta-Llama-3-8B 📄 | 45.42 | 31.1 | 69.95 | 43.91 | 36.7 |
mlabonne/OrpoLlama-3-8B-1k
corresponds to a version of this model trained on 1K samples (you can see the parameters in
this article
). The current version was trained on a full epoch.
Open LLM Leaderboard
📈 Training curves
You can find the experiment on W&B at this address .
💻 Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "mlabonne/OrpoLlama-3-8B"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Runs of mlabonne OrpoLlama-3-8B on huggingface.co
80
Total runs
0
24-hour runs
-2
3-day runs
-4
7-day runs
40
30-day runs
More Information About OrpoLlama-3-8B huggingface.co Model
Url of OrpoLlama-3-8B
OrpoLlama-3-8B huggingface.co Url
Provider of OrpoLlama-3-8B huggingface.co
Other API from mlabonne
Updated:May 19 2025
Total runs:
29.9K
Run Growth:
1.3K
Growth Rate:
4.18%
Total runs:
29.8K
Run Growth:
1.2K
Growth Rate:
3.90%
Total runs:
23.1K
Run Growth:
-8.8K
Growth Rate:
-38.22%
Total runs:
14.8K
Run Growth:
-851
Growth Rate:
-5.74%
Total runs:
9.9K
Run Growth:
-4.3K
Growth Rate:
-41.81%
Total runs:
9.0K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8.7K
Run Growth:
447
Growth Rate:
5.15%
Total runs:
8.2K
Run Growth:
552
Growth Rate:
6.74%
Total runs:
8.2K
Run Growth:
429
Growth Rate:
5.26%
Total runs:
6.7K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
3.9K
Run Growth:
958
Growth Rate:
27.99%
Total runs:
3.7K
Run Growth:
333
Growth Rate:
9.40%
Total runs:
3.0K
Run Growth:
-99
Growth Rate:
-3.35%
Total runs:
2.9K
Run Growth:
232
Growth Rate:
7.88%
Total runs:
1.8K
Run Growth:
514
Growth Rate:
28.44%
Total runs:
1.8K
Run Growth:
631
Growth Rate:
35.91%
Total runs:
1.5K
Run Growth:
-97
Growth Rate:
-6.58%
Total runs:
1.2K
Run Growth:
100
Growth Rate:
9.07%
Total runs:
1.1K
Run Growth:
601
Growth Rate:
53.66%
Total runs:
976
Run Growth:
435
Growth Rate:
44.57%
Total runs:
936
Run Growth:
-1.4K
Growth Rate:
-151.92%
Total runs:
813
Run Growth:
50
Growth Rate:
6.23%
Total runs:
712
Run Growth:
-640
Growth Rate:
-98.46%
Total runs:
669
Run Growth:
0
Growth Rate:
0.00%
Total runs:
621
Run Growth:
290
Growth Rate:
46.70%
Total runs:
619
Run Growth:
456
Growth Rate:
73.67%
Total runs:
585
Run Growth:
-93
Growth Rate:
-15.90%
Total runs:
569
Run Growth:
-85
Growth Rate:
-13.96%
Total runs:
452
Run Growth:
289
Growth Rate:
63.94%
Total runs:
402
Run Growth:
-1.6K
Growth Rate:
-409.45%
Total runs:
379
Run Growth:
-118
Growth Rate:
-31.13%
Total runs:
372
Run Growth:
-16
Growth Rate:
-4.30%
Total runs:
358
Run Growth:
0
Growth Rate:
0.00%
Total runs:
354
Run Growth:
12
Growth Rate:
3.39%
Total runs:
324
Run Growth:
-7
Growth Rate:
-2.16%
Total runs:
322
Run Growth:
35
Growth Rate:
10.87%
Total runs:
311
Run Growth:
-198
Growth Rate:
-63.67%
Total runs:
308
Run Growth:
0
Growth Rate:
0.00%
Total runs:
302
Run Growth:
114
Growth Rate:
37.75%
Total runs:
271
Run Growth:
187
Growth Rate:
69.00%
Total runs:
240
Run Growth:
135
Growth Rate:
56.25%
Total runs:
235
Run Growth:
56
Growth Rate:
23.83%
Total runs:
204
Run Growth:
-54
Growth Rate:
-26.47%
Total runs:
176
Run Growth:
0
Growth Rate:
0.00%
Total runs:
162
Run Growth:
80
Growth Rate:
49.38%
Total runs:
159
Run Growth:
84
Growth Rate:
52.83%
Total runs:
158
Run Growth:
64
Growth Rate:
40.51%
Total runs:
157
Run Growth:
-829
Growth Rate:
-528.03%
Total runs:
157
Run Growth:
17
Growth Rate:
10.83%
Total runs:
134
Run Growth:
-132
Growth Rate:
-96.35%
Total runs:
131
Run Growth:
-354
Growth Rate:
-270.23%
Total runs:
128
Run Growth:
43
Growth Rate:
33.59%
Total runs:
116
Run Growth:
63
Growth Rate:
54.31%
Total runs:
115
Run Growth:
48
Growth Rate:
41.74%
Total runs:
111
Run Growth:
-49
Growth Rate:
-44.14%
Total runs:
106
Run Growth:
4
Growth Rate:
3.74%
Total runs:
105
Run Growth:
-136
Growth Rate:
-127.10%
Total runs:
103
Run Growth:
19
Growth Rate:
18.45%
Total runs:
98
Run Growth:
-316
Growth Rate:
-322.45%
Total runs:
93
Run Growth:
-24
Growth Rate:
-25.81%
Total runs:
89
Run Growth:
-178
Growth Rate:
-200.00%
Total runs:
85
Run Growth:
-444
Growth Rate:
-522.35%
Total runs:
72
Run Growth:
51
Growth Rate:
70.83%
Total runs:
69
Run Growth:
-23
Growth Rate:
-33.33%
Total runs:
58
Run Growth:
-40
Growth Rate:
-68.97%
Total runs:
56
Run Growth:
21
Growth Rate:
37.50%
Total runs:
53
Run Growth:
-16
Growth Rate:
-30.19%
Total runs:
52
Run Growth:
-2
Growth Rate:
-3.85%
Total runs:
49
Run Growth:
0
Growth Rate:
0.00%
Total runs:
46
Run Growth:
18
Growth Rate:
39.13%
Total runs:
45
Run Growth:
0
Growth Rate:
0.00%
Total runs:
39
Run Growth:
-19
Growth Rate:
-46.34%
Total runs:
39
Run Growth:
0
Growth Rate:
0.00%
Total runs:
39
Run Growth:
12
Growth Rate:
30.77%