A comprehensive multi-model OCR benchmarking tool that evaluates Claude, Gemini, and PaddleOCR using fuzzy line-level scoring and automated reporting.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install ocr-benchmark
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install ocr-benchmark using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is OCR Benchmark?
OCR Benchmark is a specialized utility within the Openclaw Skills ecosystem designed to bring empirical rigor to optical character recognition tasks. It allows developers to simultaneously test images against various high-performance models including Claude (Opus, Sonnet, Haiku), Gemini (Pro, Flash), and PaddleOCR to determine which engine provides the best accuracy for specific document types.
By moving beyond simple text extraction, this tool provides a structured framework for evaluating OCR quality, cost-effectiveness, and latency. It specifically addresses the needs of developers working with complex layouts, such as product packaging or dense documents, ensuring that model selection is based on data-driven rankings rather than assumptions.
OCR Benchmark Use Cases
- Comparing OCR accuracy across multiple LLM providers using a single interface.
- Evaluating the cost-to-performance ratio of different model tiers for high-volume processing.
- Testing the reliability of new vision models on existing image datasets.
- Generating professional PPT reports to demonstrate OCR quality to stakeholders.
- Scoring Chinese and English text extraction on complex retail packaging.
How OCR Benchmark Works
- Collect and organize local image files in formats such as JPG, PNG, or WEBP.
- Run the benchmarking script which interfaces with providers like AWS Bedrock and Google AI Studio to extract text.
- Create or provide a ground truth JSON file representing the verified correct text for each image.
- Execute the scoring engine to apply fuzzy line-level matching using Levenshtein edit distance.
- Review the ranked terminal output or generate a comprehensive PowerPoint report for final analysis.
OCR Benchmark Setup
Navigate to the skill directory and install the necessary Python dependencies:
cd ~/.openclaw/workspace/skills/ocr-benchmark/ocr-benchmark
pip install -r requirements.txt
Configure your environment variables for the specific providers you wish to benchmark:
export AWS_REGION=us-west-2
export GOOGLE_API_KEY=your_google_key
export PADDLEOCR_ENDPOINT=https://your-endpoint
OCR Benchmark Data Schema & Taxonomy
The skill generates structured JSON files for every extraction and comparison. The primary data outputs include:
| File Type | Description |
|---|---|
{image}.{model}.json |
Individual extraction results including text, latency, and token usage. |
scores.json |
Aggregated scoring data comparing model outputs against ground truth. |
report.pptx |
Visualized benchmark results ready for presentation. |
Each extraction JSON includes metadata such as latency_seconds, input_tokens, and output_tokens to facilitate cost analysis.
OCR Benchmark Advanced Features
- Fuzzy line-level matching logic that accounts for minor punctuation and case differences.
- Automated model skipping that allows the benchmark to continue even if specific API credentials are missing.
- Score-only execution mode to refine accuracy metrics without re-running expensive API calls.
- Comprehensive normalization of text to strip whitespace and common non-semantic symbols for cleaner comparisons.
- Multi-model ranking that classifies matches as Exact, Close, Partial, or Miss.
SKILL.md
Loading
Related Openclaw Skills
Financial Analysis Agent
A professional-grade financial analysis skill for AI agents designed to automate complex accounting workflows and produce defensible, CFO-ready reports.
Eastmoney Financial Tools
A comprehensive financial data suite for stock screening, news retrieval, and real-time market analysis across global exchanges.
Develop Secure Smart Contracts with OpenZeppelin
A specialized skill for AI agents to integrate audited OpenZeppelin security patterns and token standards into blockchain projects.
March Madness AI Bracket Skill
A specialized skill for AI agents to predict, submit, and manage 63-pick NCAA March Madness tournament brackets on the maincharacter.enterprises platform.
Agent Prompt Patterns
A collection of battle-tested prompt patterns designed to move AI agents from unstable demos to reliable, production-grade automated systems.
John Smart: The Lobster Stock Analysis Assistant
A professional AI stock selection assistant that combines deep market analysis with a calm, risk-conscious investment philosophy inspired by the resilience of a lobster.
Featured*
