ewin-reg / MiniCPM5-2B-RotSVDMix-Quantized

MiniCPM5-2B-RotSVDMix (v13 Tapered Hierarchy)

Rot-SVD-Mix v13 is a stacked post-training quantization and low-rank residual decomposition of openbmb/MiniCPM5-2B calibrated under a strict decimal storage constraint ($< 2.000\text{ GB}$ decimal / strictly displaying 1.98 GB on the Hugging Face Hub).

Empirical Benchmark Card (100% Real PyTorch Execution)

All metrics below were directly measured from full forward passes on an active NVIDIA Tesla T4 GPU (14.56 GB VRAM) against the unquantized FP16 baseline model.

1. Physical Storage Footprint

2. Output Logit Concordance (25 Diverse Prompts)

Evaluated across 25 diverse multi-turn reasoning, coding, mathematics, science, and linguistic prompts:

3. WikiText-2 Test Perplexity (10,240 Tokens)

Evaluated on wikitext-2-raw-v1 (test split) across 20 non-overlapping sequences of length 512 ($L = 512$):

Architectural Formulation

Rot-SVD-Mix v13 decomposes weight matrices via orthogonal rotations and low-rank residual factors using a Tapered Hierarchy :

$W_{rot}=W{H}^{T}W\_\{rot\}\; =\; W\; H^T$ W ro t ​ = W H T

$\underset{Q,A,B}{\min }\parallel W_{rot}-(Q+A{B}^T){\parallel }^2\backslash min\_\{Q,\; A,\; B\}\; \backslash |W\_\{rot\}\; -\; (Q\; +\; A\; B^T)\backslash |^2$ Q , A , B min ​ ∥ W ro t ​ − ( Q + A B T ) ∥ 2

where:

• Pinnacle Boundary Layers (0 and 41) : $r_{down}=108, r_o=66, r_{gate/up}=46, r_q=32$ with $G=16$ for down_proj .
• Outer Boundary Layers (1–3 and 38–40) : $r_{down}=96, r_o=56, r_{gate/up}=38, r_q=26$ with $G=16$ for down_proj .
• Middle Bulk Layers (4–37) : $r_{down}=75, r_o=52, r_{gate/up}=36, r_q=25$ with $G=32$ for down_proj .
• Lossless Attention Channels : $k_{proj}$ and $v_{proj}$ are retained in 100% native FP16 across all 42 transformer layers.
• High-Precision Vocabulary : model.embed_tokens and lm_head are quantized in lossless Per-Token INT8 ($130,688 \times 2048$, $0.535\text{ GB}$ total).

Citation & Attribution

@misc{minicpm5_rotsvdmix_2026,
  title={MiniCPM5-2B Rot-SVD-Mix: Stacked Post-Training Quantization with Orthogonal Incoherence Rotations},
  author={Ewin-Reg and MiniCPM5-DocV Project Contributors},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/ewin-reg/MiniCPM5-2B-RotSVDMix}}
}