A specialized testing framework for validating AI agent persona presets and auditing routing performance through automated CLI scripts.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install evaluate-presets
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install evaluate-presets using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Evaluate Presets?
The Evaluate Presets skill is a technical utility designed to systematically test hat collection presets using direct CLI scripts. It is a vital component for developers building with Openclaw Skills who need to validate preset configurations, audit library quality, and ensure that new presets function correctly. By focusing on the principle that fresh context is reliability, this skill helps identify issues where multiple personas might bleed into a single execution iteration, ensuring high-quality agentic workflows and robust performance across various LLM backends.
Evaluate Presets Use Cases
- Testing preset configurations immediately after making structural changes to agent logic.
- Auditing the existing preset library to maintain high standards of quality and UX across Openclaw Skills.
- Validating that new presets correctly trigger the intended hat routing logic without context leakage.
- Detecting performance bottlenecks or infinite loops in complex agentic workflows before deployment.
How Evaluate Presets Works
- The utility loads test tasks from a YAML definition file to set the specific goals for the evaluation run.
- It generates a merged configuration that includes specific evaluation settings and session recording parameters.
- The agent is executed in background mode via shell scripts to allow for long-running, multi-hour auditing sessions.
- During execution, it captures output logs, exit codes, and event metrics into a structured .eval directory.
- The system extracts specific metrics like iteration counts and hat activation events to generate a final summary report.
Evaluate Presets Setup
To integrate this evaluation capability into your Openclaw Skills workflow, ensure you have yq and Cargo installed. Use the following commands to initiate testing:
# Evaluate a single specific preset
./tools/evaluate-preset.sh [preset-name] claude
# Run the complete suite of 12 standard presets
./tools/evaluate-all-presets.sh claude
Note: Always set the timeout to 600000ms and run in background mode when invoking via automated tools.
Evaluate Presets Data Schema & Taxonomy
Evaluation results are organized in a clear taxonomy within the .eval directory:
| Path | Description |
|---|---|
logs/<preset>/<timestamp>/output.log |
Full stdout and stderr capture for debugging. |
logs/<preset>/latest/session.jsonl |
Recorded session data for granular event analysis. |
logs/<preset>/latest/metrics.json |
Extracted performance data including iteration counts. |
results/<suite-id>/SUMMARY.md |
High-level Markdown report summarizing the success of the suite. |
Evaluate Presets Advanced Features
- Autonomous Fix Workflow: Automatically dispatch subagents to generate and implement code fixes for presets that fail evaluation.
- Routing Performance Triage: Specialized diagnostics to detect same-iteration hat switching which violates context reliability principles.
- Metric Extraction: Deep analysis of event bus publications vs. loop iterations to ensure Openclaw Skills are operating efficiently.
- Background Status Monitoring: Compatibility with TaskOutput tools to check progress on long-running evaluations without blocking the main agent.
SKILL.md
Loading
Related Openclaw Skills
Create Hat Collection
A conversational generator that transforms workflow ideas into validated, production-ready multi-agent YAML presets.
Code Task Generator
An intelligent utility that transforms raw project requirements or PDD implementation plans into structured, actionable Markdown task files for AI agents and developers.
Hybrid Training Plan
A professional fitness integration for Openclaw Skills that allows users to manage their training plans and log workouts via natural language.
ChilledSites
An AI-driven website builder that allows coding agents to generate, edit, and deploy professional landing pages and portfolios using simple text prompts.
Find Code Tasks
A specialized utility for listing and filtering code tasks within a repository to maintain a clear overview of project status.
pdd (Prompt-Driven Development)
A structured AI agent skill that converts vague project ideas into comprehensive technical specifications and step-by-step implementation plans through iterative refinement.
Featured*
