infosave / GLM-5.3-Flash-cmf

GLM-5.3-Flash Q2TP and Q4TP — universal CPU/GPU CMF

This repository contains the text tower of zai-org/GLM-5.3-Flash , converted directly from the official FP8 checkpoint to one memory-mapped CMF file. The same artifact is mmap-backed for CPU and Cortiq's portable GPU backends; inference does not require Python, PyTorch, a CUDA toolkit, or full-model GPU residency. This release was validated on CPU and Vulkan; Metal validation is still pending.

The upstream model is a 320B/18B-active native multimodal MoE. This release is text-generation only: it intentionally omits the vision tower and the optional speculative MTP block. cortiq info reports 313.33B retained text parameters, 45 trunk layers (34 linear KDA and 11 full DSA), a 154,880-token vocabulary, and 37,534 payload tensors.

hf download infosave/GLM-5.3-Flash-cmf glm-5.3-flash-q2tp.cmf --local-dir .
cortiq verify glm-5.3-flash-q2tp.cmf
cortiq run glm-5.3-flash-q2tp.cmf \
  --prompt "What is 2+2? Answer briefly." --no-think

Replace q2tp with q4tp in these commands for the larger, quality-oriented profile.

The file requires a GLM-5.3-compatible Cortiq build containing the dedicated glm5_next runtime. Older binaries reject this architecture instead of silently treating it as a generic Transformer. A packaged version will be listed here after the remaining GPU release gate.

File and integrity

SHA-256:

818d997b7c19dd0da1b3424ed12d869e8ff6fc4a187154c23b08f2efc4198cde  glm-5.3-flash-q2tp.cmf
6bd350591e1efd9dd098a189abf603467eed8387ee38d28f805e0891cedf36c6  glm-5.3-flash-q4tp.cmf

Neither file is a uniform low-bit dump. In Q2TP, routed/shared expert gate and up projections use q2tp while their down projections remain q4tp. In Q4TP, all large routed/shared expert projections use q4tp. Both profiles preserve always-active KDA/DSA attention projections and both vocabulary edges as q8_2f to protect recurrent error and input-channel outliers. Small norms, routers, convolution taps, mHC controls, and other sensitive nonlinear control tensors remain f16.

The source checkpoint's fine-grained E4M3 weights are decoded with their F32 weight_scale_inv planes using the official 128×128 tile orientation, then quantized independently. The CMF embeds the tokenizer, exact chat template, tensor directory, architecture descriptor, and per-tensor hashes.

Exact GLM-5.3 runtime

Cortiq executes the release architecture explicitly rather than approximating it with a generic decoder:

• four-stream Manifold-Constrained Hyper-Connections and final stream mixing;
• 34 KDA linear-attention layers and 11 DSA/MLA full-attention layers;
• DSA indexer pooling, selected-token cache policy, and ordinary-context exact full-attention fallback;
• three dense SwiGLU layers followed by 288-expert routed MoE layers, top-8 sigmoid routing, correction bias, scaling, and the unconditional shared expert;
• untied input embedding and lm_head , partial RoPE, and the native one-million-token architecture limit;
• the upstream Jinja chat template, including reasoning_effort and the direct-answer <think></think> path used by --no-think .

The release oracle compares real source FP8 tensors with the CMF runtime. For the first routed layer, Q4TP output has source correlation 0.997998 and relative RMS error 6.36% . The deterministic CPU gate renders the official 24-token direct-answer prompt and emits token 19 ( 4 ), then the model stop marker. cortiq verify independently checks every payload hash.

Models larger than VRAM

Neither the 108.27 GiB nor the 155.68 GiB file needs to fit in VRAM. CMF keeps weights mmap-backed in host memory and detects the available adapter budget. When dynamic pooling is enabled, routed experts use a bounded global GPU pool: resident experts run on the GPU, while cache misses are completed exactly on CPU and accumulated into the same MoE result. Q4TP automatic mode and systems without a supported adapter use the same file on CPU.

Useful controls:

cortiq gpu
CMF_GPU=0 cortiq run glm-5.3-flash-q2tp.cmf --prompt "CPU check" --no-think
CMF_GPU_VRAM_MB=16000 CMF_GLM_DYNAMIC_MOE=1 \
  cortiq run glm-5.3-flash-q2tp.cmf \
  --prompt "bounded forced GPU pool" --no-think
CMF_GPU_ADAPTER=0 cortiq run glm-5.3-flash-q2tp.cmf \
  --prompt "select adapter 0" --no-think
CMF_GLM_DYNAMIC_MOE=0 cortiq run glm-5.3-flash-q2tp.cmf \
  --prompt "force exact CPU MoE" --no-think

Automatic mode deliberately keeps both profiles on the exact CPU MoE path. On the measured host, bounded Vulkan expert pooling was slower for both Q2TP and Q4TP. Explicit CMF_GLM_DYNAMIC_MOE=1 remains available for compatibility and experimentation; CMF_GLM_DYNAMIC_MOE=0 always disables it. The GLM-specific pool is limited to 20–40% of the configured physical-memory envelope, and an explicit CMF_GLM_EXPERT_SLOTS value cannot bypass that cap.

Measured on an RTX 3090 with dual AMD EPYC 7H12 CPUs and 32 CPU worker threads:

The Q4TP CPU result varies with NUMA placement; the three-run spread is reported instead of presenting one noisy maximum. Forced Vulkan was about 3.4× slower, which is why it is not the Q4TP default.

For Q2TP, a bounded compatibility run with CMF_GPU_VRAM_MB=16000 CMF_GLM_DYNAMIC_MOE=1 completed successfully with the official-template answer 4 . Its 4,352-MB/512-slot expert pool peaked at 6,563 MiB of total sampled process VRAM on the physical 24-GB test card, safely below 16 GiB. The automatic normal-budget run also emitted 4 , used the exact CPU path, and peaked at 331 MiB of VRAM. CPU-only systems and smaller adapters remain valid through the same exact fallback. Metal will be tested from these immutable artifacts.

Scope and license

This CMF is a redistribution/quantization of the official GLM-5.3-Flash checkpoint . The upstream model card is MIT-licensed. Refer to the upstream repository for model limitations, intended use, citation, and the authoritative training and benchmark description.