Sponsored by
Vmake Labs Video Enhancer
Undi95 / Mistral-11B-OmniMix9
Model's Last Updated: November 18 2023
text-generation
Introduction of Mistral-11B-OmniMix9
Model Details of Mistral-11B-OmniMix9
Don't mind those at the moment, I need to finetune them for RP, it's just some tests.
WARNING: This model specifically need EOS token I completely forgot to put on the json files, and need to check what was the right ones trough the mix. Please don't use it like this if you really want to review it.
slices:
- sources:
- model: "/content/drive/MyDrive/CC-v1.1-7B-bf16"
layer_range: [0, 24]
- sources:
- model: "/content/drive/MyDrive/Zephyr-7B"
layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16
================================================
slices:
- sources:
- model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
layer_range: [0, 48]
- model: Undi95/Mistral-11B-OpenOrcaPlatypus
layer_range: [0, 48]
merge_method: slerp
base_model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
parameters:
t:
- value: 0.5 # fallback for rest of tensors
dtype: bfloat16
hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| arc_challenge | 0 | acc | 0.5623 | ± | 0.0145 |
| acc_norm | 0.5794 | ± | 0.0144 | ||
| arc_easy | 0 | acc | 0.8354 | ± | 0.0076 |
| acc_norm | 0.8165 | ± | 0.0079 | ||
| hellaswag | 0 | acc | 0.6389 | ± | 0.0048 |
| acc_norm | 0.8236 | ± | 0.0038 | ||
| piqa | 0 | acc | 0.8139 | ± | 0.0091 |
| acc_norm | 0.8264 | ± | 0.0088 | ||
| truthfulqa_mc | 1 | mc1 | 0.3978 | ± | 0.0171 |
| mc2 | 0.5607 | ± | 0.0155 | ||
| winogrande | 0 | acc | 0.7451 | ± | 0.0122 |
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 53.06 |
| ARC (25-shot) | 64.08 |
| HellaSwag (10-shot) | 84.24 |
| MMLU (5-shot) | 64.0 |
| TruthfulQA (0-shot) | 56.19 |
| Winogrande (5-shot) | 78.45 |
| GSM8K (5-shot) | 16.15 |
| DROP (3-shot) | 8.35 |
Runs of Undi95 Mistral-11B-OmniMix9 on huggingface.co
49
Total runs
0
24-hour runs
0
3-day runs
6
7-day runs
16
30-day runs
More Information About Mistral-11B-OmniMix9 huggingface.co Model
Url of Mistral-11B-OmniMix9
Mistral-11B-OmniMix9 huggingface.co Url
Provider of Mistral-11B-OmniMix9 huggingface.co
Other API from Undi95
Total runs:
876
Run Growth:
268
Growth Rate:
30.59%
Total runs:
804
Run Growth:
-245
Growth Rate:
-30.47%
Total runs:
802
Run Growth:
-251
Growth Rate:
-31.30%
Total runs:
800
Run Growth:
-229
Growth Rate:
-28.63%
Total runs:
729
Run Growth:
-253
Growth Rate:
-34.71%
Total runs:
719
Run Growth:
-277
Growth Rate:
-38.53%
Total runs:
717
Run Growth:
-225
Growth Rate:
-31.38%
Total runs:
713
Run Growth:
-252
Growth Rate:
-35.34%
Total runs:
704
Run Growth:
-258
Growth Rate:
-36.65%
Total runs:
691
Run Growth:
69
Growth Rate:
9.99%
Total runs:
532
Run Growth:
502
Growth Rate:
94.36%
Total runs:
386
Run Growth:
-11
Growth Rate:
-2.85%
Total runs:
310
Run Growth:
-63
Growth Rate:
-20.32%
Total runs:
284
Run Growth:
-27.0K
Growth Rate:
-9501.06%
Total runs:
228
Run Growth:
142
Growth Rate:
61.47%
Total runs:
197
Run Growth:
-825
Growth Rate:
-418.78%
Total runs:
192
Run Growth:
83
Growth Rate:
44.86%
Total runs:
190
Run Growth:
-155
Growth Rate:
-81.58%
Total runs:
182
Run Growth:
116
Growth Rate:
63.74%
Total runs:
163
Run Growth:
84
Growth Rate:
53.50%
Total runs:
157
Run Growth:
18
Growth Rate:
11.46%
Total runs:
150
Run Growth:
76
Growth Rate:
53.52%
Total runs:
146
Run Growth:
-449
Growth Rate:
-307.53%
Total runs:
141
Run Growth:
-469
Growth Rate:
-332.62%
Total runs:
141
Run Growth:
-477
Growth Rate:
-338.30%
Total runs:
139
Run Growth:
-20
Growth Rate:
-14.71%
Total runs:
138
Run Growth:
-60
Growth Rate:
-43.48%
Total runs:
138
Run Growth:
-96
Growth Rate:
-69.57%
Total runs:
135
Run Growth:
-37
Growth Rate:
-26.24%
Total runs:
134
Run Growth:
-322
Growth Rate:
-240.30%
Total runs:
134
Run Growth:
94
Growth Rate:
70.68%
Total runs:
123
Run Growth:
-314
Growth Rate:
-255.28%
Total runs:
122
Run Growth:
-477
Growth Rate:
-390.98%
Total runs:
120
Run Growth:
-485
Growth Rate:
-404.17%
Total runs:
114
Run Growth:
-109
Growth Rate:
-96.46%
Total runs:
106
Run Growth:
0
Growth Rate:
0.00%
Total runs:
103
Run Growth:
22
Growth Rate:
21.15%
Total runs:
97
Run Growth:
17
Growth Rate:
17.53%
Total runs:
95
Run Growth:
54
Growth Rate:
56.84%
Total runs:
95
Run Growth:
-354
Growth Rate:
-372.63%
Total runs:
93
Run Growth:
-64
Growth Rate:
-65.98%
Total runs:
86
Run Growth:
-71
Growth Rate:
-78.89%
Total runs:
85
Run Growth:
77
Growth Rate:
90.59%
Total runs:
84
Run Growth:
63
Growth Rate:
75.00%
Total runs:
83
Run Growth:
-42
Growth Rate:
-49.41%
Total runs:
83
Run Growth:
-9
Growth Rate:
-10.84%
Total runs:
82
Run Growth:
3
Growth Rate:
3.66%
Total runs:
82
Run Growth:
-4
Growth Rate:
-4.82%
Total runs:
73
Run Growth:
-134
Growth Rate:
-183.56%
Total runs:
72
Run Growth:
-1
Growth Rate:
-1.39%
Total runs:
70
Run Growth:
32
Growth Rate:
45.71%
Total runs:
69
Run Growth:
0
Growth Rate:
0.00%
Total runs:
65
Run Growth:
-43
Growth Rate:
-62.32%
Total runs:
64
Run Growth:
20
Growth Rate:
31.25%
Total runs:
63
Run Growth:
-35
Growth Rate:
-55.56%
Total runs:
62
Run Growth:
0
Growth Rate:
0.00%
Total runs:
58
Run Growth:
-17
Growth Rate:
-28.81%
Total runs:
58
Run Growth:
-45
Growth Rate:
-77.59%
Total runs:
57
Run Growth:
41
Growth Rate:
71.93%
Total runs:
57
Run Growth:
51
Growth Rate:
89.47%
Total runs:
56
Run Growth:
-165
Growth Rate:
-294.64%
Total runs:
55
Run Growth:
-31
Growth Rate:
-56.36%
Total runs:
55
Run Growth:
-9
Growth Rate:
-16.36%
Total runs:
54
Run Growth:
5
Growth Rate:
9.26%
Total runs:
53
Run Growth:
-13
Growth Rate:
-24.53%
Total runs:
52
Run Growth:
-1.1K
Growth Rate:
-2084.62%
Total runs:
51
Run Growth:
11
Growth Rate:
22.00%
Total runs:
50
Run Growth:
-33
Growth Rate:
-61.11%
Total runs:
49
Run Growth:
-28
Growth Rate:
-57.14%