avlp12 / MiniMax-M3-Alis-MLX-Dynamic

MiniMax-M3-Alis-MLX-Dynamic

The first MLX quantization of MiniMax-M3 that keeps the full vision-language model. Sensitivity-graded mixed-precision builds of MiniMaxAI/MiniMax-M3 ( 427B total / ~23B active MoE VL, MiniMax Sparse Attention, 1M context) for Apple Silicon, sized for 512 GiB and 256 GiB machines.

Exact parameter count, measured across all 59 bf16 source shards: 427.04B (language model 426.18B + vision tower 0.63B + patch-merge 0.19B + projector 0.05B).

Existing MLX quants of M3 are text-only extractions — the vision tower, multimodal projector and patch-merge MLP are deleted. These builds keep all of it : image and video prompts work end-to-end through mlx-vlm .

Builds (branches)

Never quantized, in every build:

• Router gates + e_score_correction_bias — fp32 (discrete top-4 expert selection)
• MSA index projections ( index_q_proj / index_k_proj ) — bf16 (they pick the top-16 attention blocks; a flipped selection reads different history, so this control path stays exact)
• Vision tower, multimodal projector, patch-merge MLP — bf16 (~1.4 GB total; multimodality is the point)

The shared-expert trick

M3 packs its always-on shared expert into the same SwitchLinear bank as the 128 routed experts, welding it to their bit-width. The shared expert sees 100% of tokens (a routed expert sees ~3%), so these builds unpack it and hold it at 8-bit while the routed bank drops to 3/6-bit — the single highest quality-per-GB lever in the model (+0.4% size). This requires a small mlx-vlm patch, upstream PR: Blaizzy/mlx-vlm#1544 .

Install & run

# until PR #1544 lands in a release:
pip install git+https://github.com/avlp12/mlx-vlm@minimax-m3-unpack-shared

# text
mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \
  --prompt "Explain MoE routing in three sentences." --max-tokens 300

# image
mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \
  --image photo.jpg --prompt "Describe this image." --max-tokens 300

# video
mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic \
  --video clip.mp4 --prompt "What happens in this clip?" --max-tokens 300

Python:

import mlx.core as mx
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("avlp12/MiniMax-M3-Alis-MLX-Dynamic")  # or revision="t512"
mx.set_wired_limit(mx.metal.device_info()["max_recommended_working_set_size"])

prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
out = generate(model, processor, prompt, image=["photo.jpg"], max_tokens=300)
print(out.text)

Sampling: MiniMax recommends temperature=1.0, top_p=0.95 . The model thinks in <mm:think>…</mm:think> before answering; budget max_tokens accordingly.

Measured quality (vs T512REF 8-bit reference)

Same-machine (M3 Ultra 512 GB), deterministic contexts (EN/KO/code/econ prose), KL on the reference's top-256 support; long-context on a 16K document exercising the sparse-attention path (MSA only activates beyond ~2.2K tokens — short evals cannot see it).

Reference itself: PPL 2.851 on the eval slice (T512 2.900, T256 2.994), NIAH 3/3, decode 20.6 tok/s.

Notes, honestly stated:

• T256's short-context KL (0.124) is dominated by a code context (0.217); prose contexts run 0.05–0.12. Target was <0.10 — close but a miss. A v2 with measured per-layer expert promotion may close it.
• Sparse block-selection overlap vs REF @16K (Jaccard on exact top-16 picks) reads low for both builds (T512 54%, T256 48%) — and nearly identically low, despite T512 being 2.9 bpw richer. Combined with 16K KL of 0.0011–0.0087 and 3/3 NIAH, this says the flips happen among near-tied blocks on repetitive text: selection differs, retrieval doesn't degrade.

Memory planning

KV cache ≈ 134 KiB/token fp16 (60L × 4 KV heads × 128d K+V, + 57 index-key caches) → ~13.7 GB per 100K tokens.

A 128 GiB build was evaluated and cut: the routed-expert floor alone (2-bit g128 ≈ 117 GB) plus essentials exceeds a 128 GiB Mac's realistic wired ceiling — it cannot load, so we won't ship it.

Provenance & method

• Converted from the bf16 release (869 GB, 59 shards, all files size-verified + sha256 sampled against the Hub) with mlx_vlm.convert + a custom per-module quant predicate.
• MTP heads: not present in the source release (verified against the weight index).
• Playbook: sensitivity-graded mixed precision, 3-lens adversarial design review, ship gates defined before conversion. Same lineage as Hy3-Alis-MLX-Dynamic and GLM-5.2-Alis-MLX-Dynamic-4.5bpw .