A quantitative evaluation and self-evolution framework designed to measure and optimize AI agent performance through structured feedback loops.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install agent-eval
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install agent-eval using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Agent Eval?
Agent Eval is a sophisticated framework built on the principle that anything that can be measured can be optimized. It integrates the Karpathy AutoResearch evaluation loop with the Phoenix Memory architecture to create a closed-loop system for AI self-evolution. By utilizing binary scoring and domain-specific checklists, it ensures that Openclaw Skills remain precise, reliable, and high-performing.
The framework provides specialized evaluation criteria for various agent types, including content writers, legal reviewers, scientific researchers, and medical assistants. It transforms subjective quality assessments into objective data points, allowing for automated daily self-assessments and weekly leadership reporting to drive continuous improvement.
Agent Eval Use Cases
- Automating the quality control of AI-generated public documents and research papers.
- Performing high-stakes legal contract reviews with specific risk-clause tagging.
- Conducting peer-review style quality checks for scientific research and data analysis.
- Implementing self-correcting workflows where agents modify their own instructions based on failed evaluation items.
- Generating weekly performance reports to monitor agent reliability and accuracy trends.
How Agent Eval Works
- Generation: The agent performs its assigned task (e.g., writing a report or analyzing data).
- Evaluation: The system runs a specific checklist of 3-6 binary (yes/no) criteria against the output.
- Scoring: A weighted score is calculated based on the importance of each passed or failed item.
- Failure Analysis: The system identifies specific points of failure and analyzes the root cause.
- Optimization: A small adjustment is made to the agent's prompt or configuration, and the task is rerun to check for score improvements.
- Evolution: Successful optimizations are retained in the agent's memory, while failures are discarded.
Agent Eval Setup
To integrate Agent Eval into your environment, follow these steps to initialize the evaluation memory and cron triggers:
# Initialize the evolution memory directory
mkdir -p memory/evolution/
# Create the tracking file for a specific agent
touch memory/evolution/<agent-id>.md
# Run the daily self-evaluation script (example)
python eval_processor.py --agent <agent-id> --date 2026-05-10
Agent Eval Data Schema & Taxonomy
Agent Eval organizes its findings using a structured memory system and weighted scoring tables. The data is primarily stored in Markdown for readability and version control.
Scoring Tiers
| Score | Grade | Action Required |
|---|---|---|
| 90+ | Excellent | Automatic Delivery |
| 70-89 | Good | Spot Check |
| 50-69 | Needs Improvement | Manual Review Required |
| <50 | Unqualified | Automatic Re-execution |
Evolution Logs
Located at memory/evolution/<agent-id>.md, containing:
- Daily Task Count: Number of missions processed.
- Average Score: Mean weighted performance.
- Passed/Failed Items: Specific checklist IDs (e.g., C1, S2, D3).
- Improvement Measures: Actionable changes applied to the agent.
Agent Eval Advanced Features
- Auto-Optimization Trigger: Automatically triggers a SOUL.md review if an agent scores below 70 for three consecutive days.
- Multi-Agent Synthesis: Weekly 'CEO' reports that aggregate performance across all active Openclaw Skills to identify systemic patterns.
- Pattern Recognition: Automatically writes persistent failure modes into a global
patterns.mdfile for cross-agent learning. - Domain Specialization: Pre-configured frameworks for Medical (GRADE evidence levels), Scholar (8-dimension research eval), and Shield (legal risk assessment).
SKILL.md
Loading
Related Openclaw Skills
OpenClaw Complete Backup & Restore Cycle
A comprehensive, production-ready workflow for backing up, reinstalling, and restoring OpenClaw configurations and Openclaw Skills.
Travel Guide
A comprehensive AI travel consultant providing trustworthy itineraries, holiday-specific advice, and scam-avoidance guides for global destinations.
Neuro-Humble
A clinical formation skill that teaches AI agents to adopt a neuro-humble posture of presence and disability justice instead of pathologizing human distress.
Lyft Insights Explorer
A comprehensive research tool for deep-diving into Lyft's corporate history, product offerings, and global transportation impact.
ClawHub Publish Mother Skill
A comprehensive automation workflow for authenticating, validating, and publishing Openclaw Skills to the ClawHub repository.
Mao Zedong Thought Analysis and Q&A
A comprehensive knowledge retrieval and analysis skill focused on the theoretical systems and historical context of Mao Zedong Thought.
Featured*
