Total runs:
357.6K
Run Growth:
-69.4K
Growth Rate:
-19.40%
Sponsored by
YouArt
mlabonne / Beyonder-4x7B-v2
Model's Last Updated: March 04 2024
text-generation
Introduction of Beyonder-4x7B-v2
Model Details of Beyonder-4x7B-v2
Beyonder-4x7B-v2
This model is a Mixture of Experts (MoE) made with mergekit (mixtral branch). It uses the following base models:
- openchat/openchat-3.5-1210
- beowolx/CodeNinja-1.0-OpenChat-7B
- maywell/PiVoT-0.1-Starling-LM-RP
- WizardLM/WizardMath-7B-V1.1
The recommended context length is 8k.
⚡ Quantized models
Thanks to TheBloke for the quantized models:
- GGUF : https://huggingface.co/TheBloke/Beyonder-4x7B-v2-GGUF
- AWQ : https://huggingface.co/TheBloke/Beyonder-4x7B-v2-AWQ
- GPTQ : https://huggingface.co/TheBloke/Beyonder-4x7B-v2-GPTQ
- EXL2 : https://huggingface.co/bartowski/Beyonder-4x7B-v2-exl2
🏆 Evaluation
Beyonder-4x7B-v2 is competitive with Mixtral-8x7B-Instruct-v0.1 on the Open LLM Leaderboard, while only having 4 experts instead of 8.
It also displays a significant improvement over the individual experts.
It also performs very well compared to other models on Nous benchmark suite. It's almost as good as the best Yi-34B fine-tune, which is a much bigger model: 24.2B parameters + only two experts are selected during inference (so ~12B) vs. 34B param.
| Model | AGIEval | GPT4All | TruthfulQA | Bigbench | Average |
|---|---|---|---|---|---|
| Beyonder-4x7B-v2 | 45.29 | 75.95 | 60.86 | 46.4 | 57.13 |
| NeuralHermes-2.5-Mistral-7B | 43.67 | 73.24 | 55.37 | 41.76 | 53.51 |
| OpenHermes-2.5-Mistral-7B | 42.75 | 72.99 | 52.99 | 40.94 | 52.42 |
| Nous-Hermes-2-SOLAR-10.7B | 47.79 | 74.69 | 55.92 | 44.84 | 55.81 |
| Nous-Hermes-2-Yi-34B | 50.27 | 76.00 | 60.34 | 46.69 | 58.33 |
AGIEval
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| agieval_aqua_rat | 0 | acc | 23.62 | ± | 2.67 |
| acc_norm | 23.62 | ± | 2.67 | ||
| agieval_logiqa_en | 0 | acc | 41.47 | ± | 1.93 |
| acc_norm | 43.01 | ± | 1.94 | ||
| agieval_lsat_ar | 0 | acc | 23.04 | ± | 2.78 |
| acc_norm | 23.48 | ± | 2.80 | ||
| agieval_lsat_lr | 0 | acc | 51.57 | ± | 2.22 |
| acc_norm | 52.94 | ± | 2.21 | ||
| agieval_lsat_rc | 0 | acc | 64.31 | ± | 2.93 |
| acc_norm | 64.68 | ± | 2.92 | ||
| agieval_sat_en | 0 | acc | 79.13 | ± | 2.84 |
| acc_norm | 79.13 | ± | 2.84 | ||
| agieval_sat_en_without_passage | 0 | acc | 43.20 | ± | 3.46 |
| acc_norm | 43.20 | ± | 3.46 | ||
| agieval_sat_math | 0 | acc | 34.55 | ± | 3.21 |
| acc_norm | 32.27 | ± | 3.16 |
GPT4All
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| arc_challenge | 0 | acc | 61.86 | ± | 1.42 |
| acc_norm | 64.51 | ± | 1.40 | ||
| arc_easy | 0 | acc | 85.06 | ± | 0.73 |
| acc_norm | 82.45 | ± | 0.78 | ||
| boolq | 1 | acc | 88.35 | ± | 0.56 |
| hellaswag | 0 | acc | 68.04 | ± | 0.47 |
| acc_norm | 85.12 | ± | 0.36 | ||
| openbookqa | 0 | acc | 37.80 | ± | 2.17 |
| acc_norm | 48.60 | ± | 2.24 | ||
| piqa | 0 | acc | 83.08 | ± | 0.87 |
| acc_norm | 83.95 | ± | 0.86 | ||
| winogrande | 0 | acc | 78.69 | ± | 1.15 |
TruthfulQA
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| truthfulqa_mc | 1 | mc1 | 44.55 | ± | 1.74 |
| mc2 | 60.86 | ± | 1.57 |
Bigbench
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| bigbench_causal_judgement | 0 | multiple_choice_grade | 58.95 | ± | 3.58 |
| bigbench_date_understanding | 0 | multiple_choice_grade | 66.40 | ± | 2.46 |
| bigbench_disambiguation_qa | 0 | multiple_choice_grade | 48.84 | ± | 3.12 |
| bigbench_geometric_shapes | 0 | multiple_choice_grade | 22.56 | ± | 2.21 |
| exact_str_match | 13.37 | ± | 1.80 | ||
| bigbench_logical_deduction_five_objects | 0 | multiple_choice_grade | 30.40 | ± | 2.06 |
| bigbench_logical_deduction_seven_objects | 0 | multiple_choice_grade | 20.57 | ± | 1.53 |
| bigbench_logical_deduction_three_objects | 0 | multiple_choice_grade | 52.00 | ± | 2.89 |
| bigbench_movie_recommendation | 0 | multiple_choice_grade | 44.40 | ± | 2.22 |
| bigbench_navigate | 0 | multiple_choice_grade | 52.10 | ± | 1.58 |
| bigbench_reasoning_about_colored_objects | 0 | multiple_choice_grade | 69.75 | ± | 1.03 |
| bigbench_ruin_names | 0 | multiple_choice_grade | 55.36 | ± | 2.35 |
| bigbench_salient_translation_error_detection | 0 | multiple_choice_grade | 23.65 | ± | 1.35 |
| bigbench_snarks | 0 | multiple_choice_grade | 77.35 | ± | 3.12 |
| bigbench_sports_understanding | 0 | multiple_choice_grade | 73.02 | ± | 1.41 |
| bigbench_temporal_sequences | 0 | multiple_choice_grade | 46.80 | ± | 1.58 |
| bigbench_tracking_shuffled_objects_five_objects | 0 | multiple_choice_grade | 22.08 | ± | 1.17 |
| bigbench_tracking_shuffled_objects_seven_objects | 0 | multiple_choice_grade | 19.03 | ± | 0.94 |
| bigbench_tracking_shuffled_objects_three_objects | 0 | multiple_choice_grade | 52.00 | ± | 2.89 |
🧩 Configuration
base_model: mlabonne/Marcoro14-7B-slerp
experts:
- source_model: openchat/openchat-3.5-1210
positive_prompts:
- "chat"
- "assistant"
- "tell me"
- "explain"
- source_model: beowolx/CodeNinja-1.0-OpenChat-7B
positive_prompts:
- "code"
- "python"
- "javascript"
- "programming"
- "algorithm"
- source_model: maywell/PiVoT-0.1-Starling-LM-RP
positive_prompts:
- "storywriting"
- "write"
- "scene"
- "story"
- "character"
- source_model: WizardLM/WizardMath-7B-V1.1
positive_prompts:
- "reason"
- "math"
- "mathematics"
- "solve"
- "count"
💻 Usage
Here's a notebook to run this model in 4-bit precision using a free T4 GPU on Google Colab.
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "mlabonne/Beyonder-4x7B-v2"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Output:
A Mixture of Experts (ME) is a machine learning technique that combines multiple expert models to make predictions or decisions. Each expert model is specialized in a different aspect of the problem, and their outputs are combined to produce a more accurate and robust solution. This approach allows the model to leverage the strengths of individual experts and compensate for their weaknesses, improving overall performance.
Runs of mlabonne Beyonder-4x7B-v2 on huggingface.co
111
Total runs
5
24-hour runs
18
3-day runs
31
7-day runs
57
30-day runs
More Information About Beyonder-4x7B-v2 huggingface.co Model
Url of Beyonder-4x7B-v2
Beyonder-4x7B-v2 huggingface.co Url
Provider of Beyonder-4x7B-v2 huggingface.co
Other API from mlabonne
Updated:May 19 2025
Total runs:
30.4K
Run Growth:
1.6K
Growth Rate:
5.40%
Total runs:
29.9K
Run Growth:
1.3K
Growth Rate:
4.18%
Total runs:
23.1K
Run Growth:
-8.8K
Growth Rate:
-38.22%
Total runs:
14.9K
Run Growth:
-773
Growth Rate:
-5.20%
Total runs:
10.3K
Run Growth:
-5.4K
Growth Rate:
-52.72%
Total runs:
8.7K
Run Growth:
444
Growth Rate:
5.11%
Total runs:
8.2K
Run Growth:
552
Growth Rate:
6.74%
Total runs:
8.2K
Run Growth:
429
Growth Rate:
5.26%
Total runs:
6.7K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
4.9K
Run Growth:
196
Growth Rate:
3.99%
Total runs:
3.5K
Run Growth:
401
Growth Rate:
11.45%
Total runs:
3.2K
Run Growth:
487
Growth Rate:
15.37%
Total runs:
3.0K
Run Growth:
-99
Growth Rate:
-3.35%
Total runs:
2.9K
Run Growth:
232
Growth Rate:
7.88%
Total runs:
1.8K
Run Growth:
631
Growth Rate:
35.91%
Total runs:
1.6K
Run Growth:
-183
Growth Rate:
-12.65%
Total runs:
1.5K
Run Growth:
-97
Growth Rate:
-6.58%
Total runs:
1.1K
Run Growth:
601
Growth Rate:
53.66%
Total runs:
1.1K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
976
Run Growth:
435
Growth Rate:
44.57%
Total runs:
936
Run Growth:
-1.4K
Growth Rate:
-151.92%
Total runs:
834
Run Growth:
137
Growth Rate:
16.43%
Total runs:
669
Run Growth:
0
Growth Rate:
0.00%
Total runs:
660
Run Growth:
-538
Growth Rate:
-81.52%
Total runs:
653
Run Growth:
65
Growth Rate:
9.95%
Total runs:
621
Run Growth:
290
Growth Rate:
46.70%
Total runs:
585
Run Growth:
-93
Growth Rate:
-15.90%
Total runs:
402
Run Growth:
-1.6K
Growth Rate:
-409.45%
Total runs:
379
Run Growth:
-118
Growth Rate:
-31.13%
Total runs:
372
Run Growth:
-16
Growth Rate:
-4.30%
Total runs:
358
Run Growth:
0
Growth Rate:
0.00%
Total runs:
354
Run Growth:
12
Growth Rate:
3.39%
Total runs:
324
Run Growth:
-7
Growth Rate:
-2.16%
Total runs:
322
Run Growth:
35
Growth Rate:
10.87%
Total runs:
311
Run Growth:
-198
Growth Rate:
-63.67%
Total runs:
308
Run Growth:
0
Growth Rate:
0.00%
Total runs:
271
Run Growth:
187
Growth Rate:
69.00%
Total runs:
247
Run Growth:
69
Growth Rate:
27.94%
Total runs:
176
Run Growth:
0
Growth Rate:
0.00%
Total runs:
172
Run Growth:
-15
Growth Rate:
-8.72%
Total runs:
163
Run Growth:
0
Growth Rate:
0.00%
Total runs:
163
Run Growth:
0
Growth Rate:
0.00%
Total runs:
162
Run Growth:
80
Growth Rate:
49.38%
Total runs:
159
Run Growth:
84
Growth Rate:
52.83%
Total runs:
158
Run Growth:
64
Growth Rate:
40.51%
Total runs:
157
Run Growth:
-829
Growth Rate:
-528.03%
Total runs:
157
Run Growth:
17
Growth Rate:
10.83%
Total runs:
138
Run Growth:
-160
Growth Rate:
-115.94%
Total runs:
131
Run Growth:
-354
Growth Rate:
-270.23%
Total runs:
128
Run Growth:
43
Growth Rate:
33.59%
Total runs:
116
Run Growth:
63
Growth Rate:
54.31%
Total runs:
115
Run Growth:
5
Growth Rate:
4.35%
Total runs:
112
Run Growth:
-151
Growth Rate:
-134.82%
Total runs:
111
Run Growth:
-49
Growth Rate:
-44.14%
Total runs:
109
Run Growth:
9
Growth Rate:
8.26%
Total runs:
104
Run Growth:
50
Growth Rate:
48.08%
Total runs:
103
Run Growth:
19
Growth Rate:
18.45%
Total runs:
98
Run Growth:
-316
Growth Rate:
-322.45%
Total runs:
93
Run Growth:
-24
Growth Rate:
-25.81%
Total runs:
89
Run Growth:
-178
Growth Rate:
-200.00%
Total runs:
85
Run Growth:
-444
Growth Rate:
-522.35%
Total runs:
72
Run Growth:
51
Growth Rate:
70.83%
Total runs:
69
Run Growth:
-23
Growth Rate:
-33.33%
Total runs:
63
Run Growth:
26
Growth Rate:
41.27%
Total runs:
58
Run Growth:
-40
Growth Rate:
-68.97%
Total runs:
54
Run Growth:
15
Growth Rate:
27.78%
Total runs:
53
Run Growth:
-16
Growth Rate:
-30.19%
Total runs:
49
Run Growth:
0
Growth Rate:
0.00%
Total runs:
48
Run Growth:
18
Growth Rate:
37.50%
Total runs:
46
Run Growth:
18
Growth Rate:
39.13%
Total runs:
45
Run Growth:
0
Growth Rate:
0.00%
Total runs:
40
Run Growth:
14
Growth Rate:
35.00%
Total runs:
39
Run Growth:
0
Growth Rate:
0.00%
Total runs:
39
Run Growth:
12
Growth Rate:
30.77%