ebowwa / moge-2-vits-normal-coreml

MoGe-2 ViT-S Normal — Core ML

Apple Silicon Core ML conversion of Microsoft/MoGe (commit 925b8ed835a7a9cdb7578ba15c658a0afc969030 ). All variants share the identical op graph (traced at 1x3x512x768, fp16 compute, macOS 15 target); they differ only in weight storage (const dequantize ops). At runtime all computation is fp32.

Variants (branch per variant)

Parity = per-scene median relative depth error vs the author's reference outputs (03 scenes, same photos/artifacts as the upstream moge infer release test), computed with this repo's harness ( --diff ). fp16/int8/pal8 match the reference; int4 (asymmetric) and int4-sym carry a small-accuracy-gap documented per scene. The int4 palettized 4-bit variant does NOT meet parity (~0.90) and is excluded. FOV: int8/pal8 within ~0.3 deg of reference.

Per-scene parity vs reference

int8 — int8 blockwise linear_symmetric, bs32

int4 — int4 blockwise linear (asymmetric), bs32

int4-sym — int4 blockwise linear_symmetric, b64

pal8 — int8 palettized (kmeans, 8-bit)

Performance (M1 MacBook Air, 16 GB, macOS 26.5.2, CoreML ALL units)

Real-hardware latency: single predict on a 1x3x512x768 input, mean of 5 runs after warmup, measured with the tooling venv (no ANE — this model's ops are arch-gated to CPU/GPU on A13/M1):

Provenance

Checkpoint SHA-256 verified against the reference RESULTS.md manifest before conversion. This repo is an artifact mirror of an MIT-licensed open model; conversion metadata in provenance.json .

Usage

import coremltools as ct
model = ct.models.MLModel("moge-2-vits-normal-coreml", compute_units=ct.ComputeUnit.ALL)
# input: image (1, 3, 512, 768) float32, RGB, 0..1
out = model.predict({"image": img})
# out["points"] (1,512,768,3) affine point map, out["normal"], out["mask"], out["metric_scale"]

Numerical post-processing (focal/shift recovery, depth map, intrinsics, FOV, UV) is pure numpy — see the accompanying harness/postprocess sources in the publisher's repo ( zoo/moge/ ).

Notes & limitations

• Apple Silicon only (Core ML mlprogram). No iOS deployment target shipped.
• No ANE network on M1/A13 (layer_norm/gelu/sdpa/linear are arch-gated); runs CPU/GPU via CoreML.
• Fixed input size 512x768 — resize before predict.
• int4: recommend for storage-sensitive CPU paths; expect the accuracy gap listed above (blockwise asymmetric linear, bs32).