This repo contains specialized MoE-quants for MiniMaxAI/MiniMax-M3. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
Additionally, I've provided a
J-Space lens
in the
lens/
folder, trained on ~128 text prompts.
|
Quant
|
Size
|
Mixture
|
PPL
|
1-(Mean PPL(Q)/PPL(base))
|
KLD
|
|
Q8_0
|
421.84 GiB (8.50 BPW)
|
Q8_0
|
5.202737 ± 0.034827
|
+0.3357%
|
0.030136 ± 0.000884
|
|
Q5_K_M
|
295.20 GiB (5.95 BPW)
|
Q8_0 / Q5_K / Q5_K / Q6_K
|
5.218907 ± 0.034969
|
+0.6475%
|
0.041986 ± 0.001016
|
|
Q4_K_M
|
246.11 GiB (4.96 BPW)
|
Q8_0 / Q4_K / Q4_K / Q5_K
|
5.282848 ± 0.035408
|
+1.8807%
|
0.068863 ± 0.000982
|
|
IQ4_XS
|
189.12 GiB (3.81 BPW)
|
Q6_K / IQ3_S / IQ3_S / IQ4_XS
|
5.657391 ± 0.038606
|
+9.1038%
|
0.151112 ± 0.001437
|
|
IQ3_S
|
148.04 GiB (2.98 BPW)
|
Q6_K / IQ2_S / IQ2_S / IQ3_S
|
6.417468 ± 0.044920
|
+23.7620%
|
0.321256 ± 0.002052
|
|
IQ2_S
|
134.01 GiB (2.70 BPW)
|
Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS
|
6.636852 ± 0.046125
|
+27.9929%
|
0.397892 ± 0.002336
|