FluidInference / siglip2-base-patch16-256-coreml

SigLIP 2 base (256 px) for Core ML

Core ML conversion of Google's SigLIP 2 google/siglip2-base-patch16-256 , with both towers: an image encoder and a text encoder, each returning an L2-normalized 768-d embedding. Labels are plain text chosen at run time, so it classifies images zero-shot. Google authored the Apache-2.0 source model; Fluid Inference converted it. The image encoder runs entirely on the Apple Neural Engine.

Packages

fp16 weights, macOS 14 / iOS 17 or newer. config.json holds the preprocessing and scoring constants:

• Image: resize to 256 × 256 (bilinear, antialiased like PIL), scale to [0, 1], then (x − 0.5) / 0.5 .
• Text: lowercase, Gemma tokenizer ( tokenizer.json ), append <eos> , pad with <pad> to 64 tokens. SigLIP was trained without an attention mask, so the padding must match.
• Scores: cos = image_embeds · text_embeds . sigmoid(logit_scale · cos + logit_bias) (112.90, −16.77) gives an independent probability per label; softmax or argmax over cos picks one label. Embed the labels once and reuse them for every image.

Accuracy

ImageNet-1k zero-shot, all 50,000 test images ( clip-benchmark/wds_imagenet1k , prompt this is a photo of {class}. ), fp16 on CPU + Neural Engine against the fp32 PyTorch model:

Google reports 79.1% with its own class names and prompts; the single-prompt protocol here is lower for both backends. Oxford-IIIT Pets test (3,669 photos, 37 breeds): Core ML 94.77%, PyTorch 94.74%, 99.89% identical.

Speed and memory

Apple M5 Pro (24 GB), macOS 27. Image encoder alone, one call at a time: 5.2 ms on CPU + Neural Engine (100% of ops on the ANE), 3.5 ms CPU + GPU, versus 19.1 ms for PyTorch fp32 on MPS.

End to end on 7,349 Oxford-IIIT Pets photos (JPEG decode, resize, encoder, scoring against 37 cached breed prompts), same photos and prompts, measured after model load:

PyTorch batch 1: 153.5 s, 48 photos/s, 3.64 GB. Reports: reports/ .

Use from Swift

FluidUse wraps both packages ( SigLIP2Manager , Swift Gemma tokenizer with token ids identical to the Python tokenizer) and includes the photo-sorting demo ImageSortDemo . Conversion scripts: FluidInference/mobius models/emb/siglip2/coreml .