0xSero / GLM-5.2-TR3-Vision

GLM-5.2-TR3-Vision

GLM-5.2-TR3-Vision combines the GLM-5.2 EXL3/Trellis TR3 3.0 bpw language checkpoint and quantized MTP-78 head with the GLM-5.2 MoonViT-3D vision tower and trained PatchMerger. It was live-tested through vLLM and a deployed Local Studio controller on four RTX PRO 6000 96 GB GPUs.

This is an experimental community composition, not an official Z.ai, Baseten, Brandon Music, Malaiwah, or VerdictAI release.

Source lineage

• Original model and license: zai-org/GLM-5.2
• EXL3/TR3 3.0 bpw text weights: brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw at revision 677eaa98b41b9f06abaad03953659c2872c22ce3
• Vision tower, projector, processor, and Glm5v configuration: baseten/GLM-5.2-Vision-NVFP4 at revision f6eab6117386a0c69152fdf272dc65bfd0254f9f
• Quantized MTP-78 overlay: malaiwah/GLM-5.2-EXL3-TR3-MTP78
• EXL3/Trellis runtime: verdictai/glm52-exl3-sparkinfer

The EXL3 language shards remain the routed-expert backbone. This repository adds vision_tower.safetensors and mm_projector.safetensors and exposes Glm5vForConditionalGeneration , with GlmMoeDsaForCausalLM as the nested language architecture. The vision payload contains 466,424,304 parameters (932,848,608 bytes).

Validated deployment profiles

Two separate recipes are saved on the deployed Local Studio controller. They intentionally use separate IDs so the fast text path is not mislabeled as multimodal.

GLM-5.2-EXL3 — text-only performance profile

EXL3-VISION — multimodal profile

Both profiles used the publisher-tested physical rank order 3,1,2,0 . Every GPU remained capped at 300 W.

The text runtime adds a one-line layout validation fix for partial DCP prefill batches. The projection input was already compacted before torch.bmm ; the validator now accepts the actual pitched stride of the 2,048-token workspace. This keeps DCP4, project-before-merge, and CUDA graphs enabled under concurrent mixed-length requests.

Measured speed: text-only

Hardware: 4 x RTX PRO 6000 96 GB, power limit 300 W/GPU. The canonical decode number is a 30-second sustained, concurrency-1 streaming window after warmup.

The text sustained run measured 63.87% MTP acceptance and 0.418 s TTFT, averaged 267.74 W/GPU, and peaked at 294 W.

Measured speed: vision-capable wrapper

The canonical C1 decoder remained above the 70 tok/s target with vision enabled. The long-context rows use a deliberately synthetic repeated-token prompt and 128 generated tokens. Their decode rates are not representative of normal chat because that prompt collapses MTP acceptance; the prefill rates are valid cold-cache measurements.

The vision wrapper is therefore coherent and fast enough in normal decode, but its long text-only prefill path remains slower than the dedicated text recipe. Use GLM-5.2-EXL3 when no image input is needed.

Functional validation

The following passed through the OpenAI-compatible API:

• algebra/reasoning response with a separate reasoning field;
• exact glm47 function selection and JSON arguments;
• tool-result continuation;
• MTP-3 acceptance with the quantized layer-78 draft;
• an actual 5120 x 2816 image payload through MoonViT and PatchMerger;
• OCR of the central headline, subheadline, and warning text;
• identification of the highlighted Workbench UI element when asked precisely;
• exact 400,000-token requests for both profiles.

Recipe metadata alone was not treated as proof of vision support.

Zero to running: vision profile

Requirements:

• Linux, Docker Engine, Compose, and NVIDIA Container Toolkit;
• four Blackwell GPUs with enough aggregate VRAM;
• roughly 300 GiB for weights plus runtime cache;
• the Hugging Face hf CLI.

curl -L \
  https://huggingface.co/0xSero/GLM-5.2-TR3-Vision/resolve/main/start.sh \
  -o start.sh
chmod +x start.sh
./start.sh

The launcher resumes the model download, builds the thin vision-plugin image on the pinned VerdictAI runtime, and serves GLM-5.2-VISION-EXL3 at http://127.0.0.1:8000 .

The exact validated overrides are:

export MAX_MODEL_LEN=400000
export GPU_MEMORY_UTILIZATION=0.96
export MAX_NUM_BATCHED_TOKENS=2048
export ENABLE_ASYNC_SCHEDULING=1
export VLLM_ENABLE_PCIE_ALLREDUCE=0
export VLLM_USE_B12X_DCP_A2A=0
export DCP_COMM_BACKEND=ag_rs
export KV_CACHE_DTYPE=fp8

CUDA graphs remain enabled; do not add eager-mode flags.

Text request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "GLM-5.2-VISION-EXL3",
    "messages": [{"role": "user", "content": "Explain why the sky is blue."}],
    "max_completion_tokens": 512
  }'

Image request

import base64
import requests

encoded = base64.b64encode(open("image.png", "rb").read()).decode()
response = requests.post(
    "http://127.0.0.1:8000/v1/chat/completions",
    json={
        "model": "GLM-5.2-VISION-EXL3",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {
                    "url": f"data:image/png;base64,{encoded}"
                }},
                {"type": "text", "text": "Describe the image precisely."},
            ],
        }],
        "max_completion_tokens": 1024,
    },
    timeout=600,
)
response.raise_for_status()
print(response.json())

Runtime assertions

Resolved architecture: Glm5vForConditionalGeneration
Resolved architecture: DeepSeekMTPModel
quantization=exl3
enforce_eager=False
Using ['PYNCCL'] all-reduce backends
EXL3 rank-sliced runtime planned: Trellis ... topk=8
GPU KV cache size: 672,296 tokens
Application startup complete

Limitations

• The vision checkpoint requires the included vLLM plugin and pinned VerdictAI-derived runtime. Generic Transformers loading is unsupported.
• Other GPU topologies are untested.
• Vision is a graft, not a jointly retrained EXL3 multimodal checkpoint.
• Long text-only prefill is slower in the VLM wrapper; use the dedicated text recipe when images are not required.
• MTP acceptance depends strongly on the prompt and output distribution.
• The PCIe link attached to GPU 0 emitted correctable AER events under heavy multi-GPU load on the validation host; no uncorrectable error or Xid was observed in these runs.

License

The source model repositories report the MIT license. Preserve upstream attribution and review each source model card before redistribution or use.