mudler / locate-anything.cpp-gguf

locate-anything.cpp — GGUF

GGUF builds of nvidia/LocateAnything-3B for locate-anything.cpp — a C++/ggml inference engine for open-vocabulary detection / visual grounding, no Python at inference time.

Brought to you by the LocalAI team.

The detections are identical to the official PyTorch implementation (the engine is parity-gated against it), and run faster on CPU. See the benchmarks .

Files

The full-precision f32 GGUF (~15 GB) is reproducible from the HF weights with scripts/convert_locateanything_to_gguf.py in the repo.

Quantization policy

Only the Qwen2 language-model matmuls ( attn_{q,k,v,o} , ffn_{gate,up,down} , lm.output ) are quantized. The MoonViT vision tower, the projector, all norms and biases, and the two host-read f32 tensors ( lm.tok_embd , vit.pos_emb ) stay f32 — so the parity-sensitive vision path is untouched. q8_0 is byte-identical on box output; lower bit-widths trade a little box precision for size.

Usage

# build the CLI (see the repo README), then:
locate-anything-cli detect \
    --model locate-anything-q8_0.gguf \
    --input image.jpg \
    --prompt "Locate all the instances that matches the following description: person</c>car." \
    --annotated out.png
# -> {"detections":[{"label":"person","box":[...]}, ...]}  + an annotated PNG

Decode modes: --mode hybrid (default), slow , fast . Separate categories in the prompt with </c> .

License

The model weights are NVIDIA's, distributed under NVIDIA's license ; this repository redistributes them in GGUF form for use with locate-anything.cpp (MIT).