inclusionAI / ArmorOCR-GGUF

ArmorOCR-GGUF

This is the GGUF quantized release of ArmorOCR — a two-stage framework for grounded adversarial OCR perception built on Qwen3-VL-8B-Instruct. It provides two quantization tiers, Q8_0 and Q4_K_M , for both the main model and its vision projector ( mmproj ), intended to be served with llama.cpp .

📖 For training details, the AdvSpot benchmark, and the full evaluation pipeline, please visit the GitHub repo .

Quickstart

Build llama.cpp with CUDA support, then serve a tier with llama-server + --mmproj :

# 1) Build llama.cpp (CUDA)
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release
export LLAMA_BIN=$(pwd)/build/bin
pip install requests tqdm

# 2) Start the OpenAI-compatible server (Q8_0 here; use Q4_K_M likewise)
bash serve_gguf.sh Q8_0 8080

# 3) Run inference against the local server (mirrors the ArmorOCR quickstart)
import base64, requests

with open("path/to/image.png", "rb") as f:
    img_url = f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode()}"

resp = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "model": "ArmorOCR-GGUF",
    "messages": [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": img_url}},
        {"type": "text", "text": ("Please identify the text in the image. "
                                  "Put your reasoning inside <analyze></analyze> "
                                  "and your final recognized text inside <answer></answer>.")},
    ]}],
    "temperature": 0.0,
    "max_tokens": 1024,
}, timeout=600)
print(resp.json()["choices"][0]["message"]["content"])

serve_gguf.sh is included in this repo for convenience.

Evaluation on AdvSpot

Region-grounded VQA accuracy and IoU on the AdvSpot benchmark. The two GGUF tiers were evaluated on a single A100 GPU with llama-server ; the base column reports the original ArmorOCR results from the paper (evaluated on PPU, same data).

The quantized checkpoints retain accuracy close to the original ArmorOCR (Q8_0 is marginally higher on Acc), with a slight drop in IoU — consistent with quantizing the vision encoder/projector.

License

Released under the Apache License 2.0 . Use is additionally subject to the license and acceptable-use policy of the base model Qwen/Qwen3-VL-8B-Instruct .

Citation

@misc{cao2026armorocrgroundedadversarialvisual,
      title={ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation}, 
      author={Linhan Cao and Siyuan Li and Jun Lan and Liangbo He and Guannan Li and Xiaolei Huang and Jun Jia and Shuheng Zhou and Huijia Zhu and Weiqiang Wang and Wei Sun},
      year={2026},
      eprint={2608.20122},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.20122}, 
}