PinchBench is a comprehensive benchmarking suite designed to measure the performance and accuracy of Openclaw Skills across various real-world tasks.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install pinchbench
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install pinchbench using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is PinchBench Benchmark Skill?
PinchBench acts as the definitive evaluation framework for Openclaw Skills, providing a standardized way to measure how effectively different LLMs power agent workflows. By executing a series of 23 diverse tasks—ranging from basic sanity checks to complex multi-step API integrations—it provides developers with actionable data on model reliability and capability. This skill ensures that your Openclaw Skills meet the highest standards of performance before deployment.
The skill integrates seamlessly with the OpenClaw ecosystem, allowing users to run local tests or contribute to a global leaderboard at pinchbench.com. Whether you are testing custom prompts or comparing different model providers, PinchBench provides the quantitative metrics needed to refine your Openclaw Skills and ensure they handle real-world scenarios like calendar management, market research, and code generation with precision.
PinchBench Benchmark Skill Use Cases
- Assessing the reasoning capabilities of different LLMs when executing Openclaw Skills.
- Validating custom Openclaw Skills using a standardized set of automated grading criteria.
- Contributing benchmark data to a public leaderboard to compare agent performance across the community.
- Identifying bottlenecks in multi-step workflows involving email triage, data analysis, or file management.
- Testing the impact of model updates or configuration changes on task success rates.
How PinchBench Benchmark Skill Works
- The user initiates a benchmark run using the CLI, specifying a target model and task suite.
- The skill loads task definitions from the tasks directory, including prompts and grading logic.
- The OpenClaw agent executes each task, interacting with various tools and APIs as required.
- Results are automatically graded based on expected behavior and automated Python functions.
- A JSON report is generated locally, summarizing scores, timeouts, and failure points.
- (Optional) Results are uploaded to the public leaderboard for global comparison.
PinchBench Benchmark Skill Setup
Ensure you have Python 3.10+ and the uv package manager installed. Navigate to the skill directory and run the following commands:
# Register for an API token to enable leaderboard submissions
uv run benchmark.py --register
# Run a full benchmark suite using a specific model
uv run benchmark.py --model anthropic/claude-sonnet-4
# Run only automated tasks to save time
uv run benchmark.py --model anthropic/claude-sonnet-4 --suite automated-only
PinchBench Benchmark Skill Data Schema & Taxonomy
PinchBench organizes benchmark data into structured JSON files within an output directory. The schema captures detailed performance metrics for every Openclaw Skills execution:
| Field | Description |
|---|---|
task_id |
Unique identifier for the benchmark task |
score |
The mean grade awarded by the automated checker |
grading |
Detailed breakdown of grading criteria and logic |
model |
The specific LLM identifier used during the run |
timestamp |
When the benchmark was executed |
Users can use tools like jq to extract specific insights, such as failed tasks or average scores across the entire suite of Openclaw Skills.
PinchBench Benchmark Skill Advanced Features
- Custom Task Creation: Developers can define new benchmarks by creating Markdown files in the tasks folder with custom YAML frontmatter and Python grading logic.
- Multi-Run Averaging: Use the
--runsflag to execute tasks multiple times, ensuring statistical reliability for your Openclaw Skills. - Automated Leaderboard Integration: Seamlessly sync local test results with the global leaderboard using the built-in upload functionality.
- Granular Task Selection: Target specific capabilities by running individual task IDs like task_01_calendar or task_18_market_research.
- Timeout Scaling: Adjust the
--timeout-multiplierto accommodate slower models while maintaining fair testing conditions for all Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
M Valuation Method
A professional stock valuation framework using ROIC and CAPM to determine intrinsic value and investment risk.
StonebornBot NFT Minting Agent
A high-performance NFT minting bot for Ethereum and EVM chains designed for sub-100ms execution and multi-wallet orchestration.
Clawland
Clawland enables AI agents to play on-chain odd/even games on the Solana devnet by minting GEM tokens from SOL or USDC.
React Native Logs CLI
A lightweight CLI tool for reading React Native Metro logs via CDP, designed for seamless integration with Openclaw Skills.
GSD (Get Shit Done) Project Workflow
GSD is a comprehensive AI-driven workflow for taking projects from initial concept to verified execution through systematic planning and parallelized research.
GSD (Get Shit Done): Project Planning & Execution
A complete end-to-end AI agent workflow for taking projects from initial idea to verified execution through systematic planning and research.
Featured*
