orcarouter / GLM-5.3-MLX

---

An MLX build of the official zai-org/GLM-5.3 — Z.AI's 753B-total / ~39B-active MoE flagship for frontier coding and long-horizon agentic engineering, built on the glm_moe_dsa architecture (MLA + DeepSeek-style sparse attention with a shared indexer , 1M context) — quantized to MLX for Apple Silicon and the MLX CUDA backend. Four precisions ( 2 / 3 / 4 / 6-bit ) ship as subfolders with the 4-bit build mirrored at the repo root , produced with OrcaSAQ (Sensitivity-Aware Quantization) : calibration-free and architecture-aware, giving sensitive tensors more bits (shared experts +2 , down_proj +1 ) while attention stays at 8-bit and the DSA indexer stays in BF16 in every build. Browse all models in the OrcaRouter Model Catalog ; deployed as API here . Put this model to work reviewing your pull requests with OrcaCode Review .

---

Available quantizations

The repository root is a mirror of 4-bit/ , so orcarouter/GLM-5.3-MLX resolves to the recommended default with no subfolder needed.

Quantized from the official FP8 release (756 GB, block-wise e4m3 128×128, dynamic activation scheme). Bit-widths above are the expert base precision; the OrcaSAQ policy below raises sensitive tensors above the base. Group size is 64 everywhere except the 2-bit routed experts, which use 32 — at two bits the finer grid is what keeps the build usable (see the fidelity table).

Hardware reality check. This is a 753B model; even the smallest build is 322 GB. On a 512 GB M3 Ultra Mac Studio , 2-bit and 3-bit leave comfortable KV headroom and 4-bit fits but runs tight — raise the wired-memory limit and keep the context bounded. 6-bit does not fit any single Apple Silicon machine today: run it across two 512 GB boxes with mlx.distributed , or on a CUDA host (8×H200 = 1128 GB) via the MLX CUDA backend. If the hardware isn't there, the hosted API needs no local memory at all.

No 8-bit build was produced: 6-bit already lands at cosine ≥ 0.9997 against FP8, and 8-bit expert weights would exceed the FP8 source in size.

---

OrcaSAQ — Sensitivity-Aware Quantization

OrcaSAQ is a calibration-free, architecture-aware mixed-precision quantization method designed to preserve model quality while aggressively reducing memory footprint.

Instead of running expensive calibration datasets or per-layer sensitivity searches, OrcaSAQ uses architectural and tensor-role priors to determine which weights are most sensitive to quantization. Critical components receive higher precision, while more quantization-tolerant tensors remain at the target base precision.

For MoE architectures, for example, OrcaSAQ can preserve shared experts at base +2 bits , down_proj at base +1 bit , keep gate_proj and up_proj at the base precision, and retain sensitive non-quantized components at their source dtype.

The result is a fast, deterministic, zero-calibration quantization pipeline that can be applied to new architectures without costly dataset-driven optimization.

Architecture-aware. Calibration-free. Quality-preserving.

Two properties of this checkpoint drive the whole recipe:

• Attention is FP8 upstream (unlike GLM-5.3-Flash, where it shipped BF16), so it is quantizable — and it is pinned to 8-bit in every build , never dropped to the base tier.
• The DSA indexer is kept in BF16 everywhere. It selects which tokens attention attends to, so it sits directly on the long-context path and is left at full precision.

Everything that was not FP8 in the base release — the indexer, router gate weights, the FP32 e_score_correction_bias , every norm, embed_tokens and lm_head — is carried through at its original dtype , never a lossy cast.

Bit allocation for GLM-5.3

58,224 tensors are quantized in total (97.5 % of all parameters live in the routed experts). Bits are rounded to the nearest MLX-supported width ( {2,3,4,5,6,8} ), and group size must divide the tensor's last dim — gs64 on attention is mandatory because qk_nope_head_dim is 192 and gs128 breaks mlx-lm's kv_b → embed_q re-quantization.

Effective width on the experts, including the FP16 scales and biases MLX stores per group:

The assignment is recorded in config.json → quantization (and mirrored to quantization_config ) as per-module {group_size, bits} overrides keyed by MLX module path — e.g. model.layers.3.mlp.switch_mlp.down_proj . MLX fuses a layer's 256 routed experts into one switch_mlp , so 699 entries cover all 58,224 tensors. This is what the loader reads at load time and it is required : the top-level bits / group_size alone would mis-shape every raised-precision tensor. A full per-tensor record is also shipped as quantization_map.json .

The MTP (multi-token prediction) layer — layer 78 — is dropped , and num_nextn_predict_layers is set to 0 . 791 tensors (~9.95B params) are removed; MLX runtimes drop it at load anyway.

---

Quality vs FP8

All three tables compare each build against the full FP8 reference (dequantized to BF16 and run through the identical glm_moe_dsa forward, so the only variable is the quantization). Sizes are decimal GB, matching the file sizes in this repo.

Perplexity

Measured against the FP8 reference on wikitext-2 test, 4 chunks × 1024 tokens (4,092 predicted tokens). Both sides run the identical glm_moe_dsa forward on the same FP8 kernels — each build's actual dequantized bytes are injected back into the transformers model — so quantization is the only variable.

6-bit lands within noise of the FP8 reference — read it as indistinguishable.

KL divergence & Top-1 token agreement

KLD is KL(ref ‖ quant) per token against the FP8 reference distribution, Top-1 is how often the build's argmax matches the reference's. Lower KLD and higher Top-1 = closer to the original. Same corpus and token budget as the PPL table.

All three measurements — weight fidelity, perplexity, and KLD/Top-1 — agree on the ordering: 6-bit is effectively lossless, 4-bit costs about 2 % perplexity for 43 % less memory and is the recommended default, and 3-bit holds up well at 40 % under FP8. 2-bit is the memory-first option — reach for it when the hardware cannot hold anything larger.

Weight-space fidelity — measured on every quantized tensor at pack time (58,224 per build):

Worst-case per build ( cos_min / snr_min ): 6-bit 0.99972 / 32.5 dB · 4-bit 0.99509 / 20.1 dB · 3-bit 0.97869 / 13.7 dB · 2-bit 0.92718 / 8.3 dB.

Relative error is the mean of the per-tensor relative error 10^(-SNR/20) over all 58,224 quantized tensors — not a value back-derived from the mean SNR, which would understate it (1.4 / 7.1 / 14.7 / 27.5 %).

Per role — this is where the OrcaSAQ policy shows up:

Cosine distribution across all quantized tensors (tight percentiles = no outlier tensors hiding behind a good mean):

The worst tensors in every build are the same ones — early-layer routed experts ( layers.3–5 , e.g. layers.4.mlp.experts.168.gate_proj ) — and 6-bit still holds them above 0.9997.

Build verification — every build passed a structural release gate before upload:

• accounting — 175,242 tensors on disk, 58,224 quantized modules, weight / scales / biases triplets all paired
• dtype preservation — kept tensors byte-identical in their source dtype (FP32 router bias stays FP32; FP8 keep goes through a proper block dequant, never a raw cast)
• index integrity — model.safetensors.index.json resolves every tensor; total_size matches the shards on disk
• config ↔ map reconstruction — the 699 per-module overrides reproduce the full per-tensor map exactly
• spot dequant — randomly sampled tensors per role dequantized and compared against the golden FP8 values
• runtime skeleton — mlx-vlm's glm_moe_dsa module tree vs the post-sanitize key set: 1497/1497 , zero missing, zero extra

---

---

Usage

Hosted API — no download

The full-precision model is live on OrcaRouter as z-ai/glm-5.3 — 1M context, 128K max output, $1.26 / $3.96 per 1M input / output tokens:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.orcarouter.ai/v1",
    api_key=os.environ["ORCAROUTER_API_KEY"],
)

response = client.chat.completions.create(
    model="z-ai/glm-5.3",
    messages=[{"role": "user", "content": "Refactor this module and explain the tradeoffs."}],
)
print(response.choices[0].message.content)

https://www.orcarouter.ai/models/z-ai/glm-5.3

Run it locally (mlx-vlm)

Use mlx-vlm , not mlx-lm . GLM-5.3 is a text-only model, but as of mlx-lm 0.31.3 the glm_moe_dsa implementation builds an indexer for every layer, while this checkpoint shares one indexer across four layers ( indexer_types ) — 285 tensors come up missing. mlx-vlm 0.6.17 implements the shared-indexer layout and loads these builds as-is. Check for mlx-lm support before switching back.

pip install -U "mlx-vlm>=0.6.17"

The repo root is the 4-bit build, so the recommended default needs no subfolder — the excludes are what stop hf download from also pulling the four variant folders:

hf download orcarouter/GLM-5.3-MLX --local-dir ./GLM-5.3-MLX \
    --exclude "2-bit/*" "3-bit/*" "4-bit/*" "6-bit/*"

python -m mlx_vlm.generate --model ./GLM-5.3-MLX \
    --prompt "Write a Rust function that merges overlapping intervals." --max-tokens 512

To pick a different precision, download just that subfolder and load it from the local path:

hf download orcarouter/GLM-5.3-MLX --include "6-bit/*" --local-dir ./GLM-5.3-MLX

python -m mlx_vlm.generate --model ./GLM-5.3-MLX/6-bit \
    --prompt "Write a Rust function that merges overlapping intervals." --max-tokens 512

Python API:

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("./GLM-5.3-MLX")          # repo root == 4-bit
prompt = apply_chat_template(processor, model.config, "Explain MLA vs GQA in three sentences.")

print(generate(model, processor, prompt, max_tokens=512, verbose=True))

Reasoning effort. GLM-5.3's chat template takes a reasoning_effort variable — low , high , or max (the default when unset) — emitted as a system directive, plus a clear_thinking flag that strips prior <think> blocks from the history. Both pass straight through apply_chat_template :

prompt = apply_chat_template(processor, model.config, "Port this service to async.",
                             reasoning_effort="low")

Tool calling is in the template too ( <tool_call> / <arg_key> blocks, tools=[...] ).

Long context. max_position_embeddings is 1,048,576 ( rope_theta 8e6). The KV cache, not the weights, is what will bite you at those lengths — budget for it, or cap it with --max-kv-size / quantize it with --kv-bits .

On Apple Silicon the Metal backend is used automatically. On the Linux CUDA backend install mlx[cuda] and set MLX_CUDA_USE_CUDNN_SDPA=0 .

---

🐳 Run it with OrcaCode Review

Website · GitHub

Models are only half the system.

OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:

• reviews every PR
• finds security + correctness issues
• posts inline findings
• P0/P1 can block merges
• swap models anytime

Open model. Open harness. Open bill.

• Website — https://www.orcarouter.ai
• GitHub — https://github.com/Continuum-AI-Corp/Orca-Code-Review

---

Build Provenance

Each folder carries its own build_manifest.json (exact recipe, source path, dropped-tensor count, toolchain versions) and fidelity_summary.json (per-role cosine/SNR with the five worst tensors named), so any claim in this card can be checked against the build itself.

---

GLM-5.3

Available full-precision on the OrcaRouter API as z-ai/glm-5.3 — https://www.orcarouter.ai/models/z-ai/glm-5.3

---

GLM-5.3 Highlights

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks:

• Stronger coding: the most capable open-weights model for coding, with a 50 % improvement over GLM-5.2 on Z.AI's in-house Code Bench, and open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam.
• Emergent cyber capability: state of the art on CyberGym for vulnerability discovery, with the largest gains further up the exploitation chain — more than doubling GLM-5.2 on exploitation benchmarks.

---

Official benchmarks

Reported by Z.AI for the full-precision model. These are the upstream model's numbers, not measurements of these MLX builds — see Quality vs FP8 above for how each quantization tracks the FP8 release.

Full evaluation protocols and footnotes are in the official model card .

---

Model Overview

• Type: Mixture-of-Experts Causal LM ( glm_moe_dsa , GlmMoeDsaForCausalLM )
• Parameters: 753B total · ~39B active per token (22.6B routed + 16.7B always-on; 743B after the MTP layer is dropped)
• Experts: 256 routed + 1 shared, top-8 routing, sigmoid scoring with e_score_correction_bias , single group, routed_scaling_factor 2.5
• Layers: 78 (3 dense + 75 MoE), plus 1 MTP layer (dropped in these builds)
• Attention: MLA — q_lora_rank 2048, kv_lora_rank 512, 64 heads, qk_nope 192 + qk_rope 64, v_head_dim 256 — with a DeepSeek-style sparse-attention indexer ( index_topk 2048, 32 index heads) built on every 4th layer and shared by the three that follow
• Precision (base release): FP8 (block-wise e4m3 , 128×128), dynamic activation scheme
• Context: 1,048,576 tokens · vocab 154,880
• Modality: text

---

Best Practices

• Pick a precision: 6-bit for near-lossless, 4-bit (repo root) as the everyday default, 3-bit when memory is the binding constraint, 2-bit when nothing else fits — the tables above quantify the trade at each step.
• Sampling: follow the official GLM-5.3 guidance. The shipped generation_config.json is temperature 1.0, top_p 0.95; long-horizon agentic and coding tasks want generous max-tokens headroom.
• Reasoning effort: leave it at the default max for hard engineering work; drop to low for latency-sensitive turns.
• Long context: budget KV, not just weights — at 1M tokens the cache dwarfs the difference between two of these builds.
• Or skip the download: the unquantized model is served at full precision on the OrcaRouter API as z-ai/glm-5.3 .

---

Citation

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team and Aohan Zeng and Xin Lv and Zhenyu Hou and Zhengxiao Du and others},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}

License

Inherited from zai-org/GLM-5.3 — an MIT-style grant (use, modify, distribute, fine-tune, sell) with one added condition: operators of a Model-as-a-Service business whose aggregate revenue exceeds US$10B over any 12 consecutive months must pass a Z.AI security review before commercial use. The full text ships as LICENSE in this repo. Quantization does not change the underlying license obligations.