Total runs:
1.2K
Run Growth:
212
Growth Rate:
17.01%
Sponsored by
APIMart
CalamitousFelicitousness / SorcererLM-8x22b-FP8-Dynamic
Model's Last Updated: October 04 2024
Introduction of SorcererLM-8x22b-FP8-Dynamic
Model Details of SorcererLM-8x22b-FP8-Dynamic
SorcererLM-8x22b-bf16
This repo contains the copy of the original quantized to FP8. Original: rAIfle/SorcererLM-8x22b-bf16
Oh boy, here we go. Low-rank (
r=16, alpha=32
) 16bit-LoRA on top of
WizardLM-2-8x22B
, trained on 2 epochs of (cleaned & deduped) c2-logs. As far as I can tell, this is an upgrade from
WizardLM-2-8x22B
for RP purposes.
Alongside this ready-to-use release I'm also releasing
the LoRA itself
as well as
the earlier
epoch1
-checkpoint of the LoRA
.
Why A LoRA?
The choice was fully intentional. I briefly considered a FFT but for this particular use-case a LoRA seemed a better fit.
WizardLM-2-8x22B
is smart by itself but its used vocabulary leaves much to be desired when it comes to RP. By training a low-rank LoRA on top of it to teach it some of Claude's writing style, we remedy that.
Prompting
-
Use the templates in
Quant-Cartel/Recommended-Settings
under the
SorcererLM-folder. - Or Vicuna 1.1 and a sane context template. It's somewhat sensitive to samplers, I'd recommend Temperature 1, MinP 0.05 and a dash of DRY but YMMV. Shorter prompts seem to work better, too.
Quantized Versions
Acknowledgments
The main shoutout I want to make is to my Cartel bros, Envoid and particularly I^2 , for being amazing. I count this as a team effort, so they deserve kudos too if you like this.
Training
Trained using
qlora-pipe
. Configs included in the
train
-subfolder.
Safety
... n/a
Runs of CalamitousFelicitousness SorcererLM-8x22b-FP8-Dynamic on huggingface.co
9
Total runs
1
24-hour runs
2
3-day runs
0
7-day runs
4
30-day runs
More Information About SorcererLM-8x22b-FP8-Dynamic huggingface.co Model
Url of SorcererLM-8x22b-FP8-Dynamic
SorcererLM-8x22b-FP8-Dynamic huggingface.co Url
Provider of SorcererLM-8x22b-FP8-Dynamic huggingface.co
Other API from CalamitousFelicitousness
Updated:April 09 2026
Total runs:
1.2K
Run Growth:
203
Growth Rate:
16.71%
Total runs:
394
Run Growth:
70
Growth Rate:
17.77%
Total runs:
391
Run Growth:
36
Growth Rate:
9.21%
Total runs:
280
Run Growth:
-357
Growth Rate:
-120.61%
Total runs:
213
Run Growth:
1
Growth Rate:
0.47%
Total runs:
195
Run Growth:
-833
Growth Rate:
-427.18%
Total runs:
182
Run Growth:
27
Growth Rate:
14.84%
Total runs:
160
Run Growth:
-757
Growth Rate:
-473.13%
Total runs:
153
Run Growth:
-32
Growth Rate:
-20.65%
Total runs:
145
Run Growth:
-218
Growth Rate:
-149.32%
Total runs:
94
Run Growth:
60
Growth Rate:
69.77%
Total runs:
52
Run Growth:
-186
Growth Rate:
-357.69%
Total runs:
47
Run Growth:
9
Growth Rate:
19.57%
Total runs:
45
Run Growth:
13
Growth Rate:
28.89%
Total runs:
42
Run Growth:
-122
Growth Rate:
-290.48%
Total runs:
40
Run Growth:
-46
Growth Rate:
-115.00%
Total runs:
40
Run Growth:
-28
Growth Rate:
-70.00%
Total runs:
39
Run Growth:
-13
Growth Rate:
-33.33%
Total runs:
38
Run Growth:
9
Growth Rate:
24.32%
Total runs:
36
Run Growth:
22
Growth Rate:
62.86%
Total runs:
28
Run Growth:
8
Growth Rate:
28.57%
Total runs:
28
Run Growth:
21
Growth Rate:
75.00%
Total runs:
26
Run Growth:
20
Growth Rate:
76.92%
Total runs:
25
Run Growth:
18
Growth Rate:
72.00%
Total runs:
23
Run Growth:
21
Growth Rate:
91.30%
Total runs:
21
Run Growth:
15
Growth Rate:
71.43%
Total runs:
21
Run Growth:
-17
Growth Rate:
-70.83%
Total runs:
20
Run Growth:
-29
Growth Rate:
-145.00%
Total runs:
19
Run Growth:
9
Growth Rate:
47.37%
Total runs:
19
Run Growth:
8
Growth Rate:
42.11%
Total runs:
16
Run Growth:
2
Growth Rate:
12.50%
Total runs:
16
Run Growth:
1
Growth Rate:
6.25%
Total runs:
16
Run Growth:
6
Growth Rate:
37.50%
Total runs:
15
Run Growth:
11
Growth Rate:
73.33%
Total runs:
15
Run Growth:
5
Growth Rate:
33.33%
Total runs:
15
Run Growth:
1
Growth Rate:
6.67%
Total runs:
14
Run Growth:
-1
Growth Rate:
-7.14%
Total runs:
14
Run Growth:
10
Growth Rate:
71.43%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
14
Run Growth:
2
Growth Rate:
14.29%
Total runs:
13
Run Growth:
11
Growth Rate:
84.62%
Total runs:
13
Run Growth:
-1
Growth Rate:
-7.69%
Total runs:
12
Run Growth:
9
Growth Rate:
75.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
5
Growth Rate:
45.45%
Total runs:
10
Run Growth:
-4
Growth Rate:
-40.00%
Total runs:
10
Run Growth:
1
Growth Rate:
10.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
-4
Growth Rate:
-44.44%
Total runs:
9
Run Growth:
2
Growth Rate:
22.22%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
-1
Growth Rate:
-12.50%
Total runs:
8
Run Growth:
-2
Growth Rate:
-25.00%
Total runs:
8
Run Growth:
2
Growth Rate:
25.00%
Total runs:
7
Run Growth:
-2
Growth Rate:
-28.57%
Total runs:
7
Run Growth:
1
Growth Rate:
14.29%
Total runs:
7
Run Growth:
-5
Growth Rate:
-71.43%
Total runs:
6
Run Growth:
0
Growth Rate:
0.00%
Total runs:
6
Run Growth:
3
Growth Rate:
50.00%
Total runs:
6
Run Growth:
1
Growth Rate:
16.67%
Total runs:
6
Run Growth:
0
Growth Rate:
0.00%
Total runs:
6
Run Growth:
-2
Growth Rate:
-33.33%
Total runs:
6
Run Growth:
-1
Growth Rate:
-16.67%
Total runs:
6
Run Growth:
-1
Growth Rate:
-16.67%
Total runs:
5
Run Growth:
-5
Growth Rate:
-100.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
1
Growth Rate:
20.00%
Total runs:
5
Run Growth:
-3
Growth Rate:
-60.00%
Total runs:
4
Run Growth:
-5
Growth Rate:
-125.00%
Total runs:
4
Run Growth:
2
Growth Rate:
50.00%
Total runs:
4
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1
Run Growth:
-1
Growth Rate:
-100.00%
Total runs:
1
Run Growth:
-1
Growth Rate:
-100.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%