An automated evaluation engine that scores miner submissions based on content consistency and structured data quality.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install dataset-evaluation
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install dataset-evaluation using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Dataset Evaluation Skill?
The Dataset Evaluation skill is a specialized tool designed for Openclaw Skills to provide objective, multi-layered assessment of data submissions. It serves as a rigorous gatekeeper for data integrity, ensuring that raw content is both internally consistent and accurately transformed into structured formats. By analyzing five distinct data samples against a target JSON schema, the skill eliminates manual oversight and provides a deterministic miner score.
This skill is essential for decentralized data collection and processing networks where quality control is paramount. It bridges the gap between raw text ingestion and structured database entry by employing weighted scoring algorithms that account for text similarity, field completeness, and value accuracy. Integrating this into your Openclaw Skills workflow ensures high-fidelity datasets with minimal hallucination risks.
Dataset Evaluation Skill Use Cases
- Validating miner submissions in decentralized data labeling or scraping networks.
- Automating quality assurance for LLM fine-tuning datasets.
- Verifying the accuracy of OCR or transcription outputs against structured schemas.
- Auditing data consistency across multiple samples of the same underlying content.
How Dataset Evaluation Skill Works
- Content Normalization: The skill receives 5 cleaned data samples and strips HTML, converts text to lowercase, and removes excessive whitespace.
- Consistency Analysis: It computes pairwise similarity using cosine or Jaccard metrics to ensure all samples represent the same content (40% weight).
- Schema Validation: The structured JSON is compared against the dataset schema to verify required fields and data types.
- Accuracy Verification: Individual field values are cross-referenced with the cleaned text to ensure the extracted data is factual and present in the source.
- Weighted Scoring: A final miner score is calculated by combining the content consistency (40%) and structuring quality (60%) scores.
Dataset Evaluation Skill Setup
To integrate this skill into your Openclaw Skills environment, ensure your agent has access to the evaluation logic and a similarity metric library.
# Install the evaluation skill via openclaw CLI
openclaw install dataset_evaluation
# Configure your evaluation parameters
openclaw configure dataset_evaluation --threshold 0.8
Dataset Evaluation Skill Data Schema & Taxonomy
The skill processes a specific input object and returns a detailed scoring report.
Input Schema
| Field | Type | Description |
|---|---|---|
| cleaned_data_list | Array | 5 strings of cleaned source text |
| structured_data | Object | The JSON output generated by the miner |
| dataset_schema | Object | The blueprint defining required fields and types |
Output Report
- miner_score: The final weighted average (0-100).
- content_consistency_score: Score based on text similarity.
- structuring_quality_score: Composite score of completeness, accuracy, and type correctness.
Dataset Evaluation Skill Advanced Features
- Deterministic Scoring: Ensures that the same input always yields the same quality score.
- Multi-Metric Evaluation: Combines embedding-based cosine similarity with hard schema validation.
- Granular Deductions: Specifically penalizes information insufficiency if the miner misses obvious data points present in the text.
- Extensible Schema Support: Works with any custom JSON schema provided in the input payload.
SKILL.md
Loading
Related Openclaw Skills
Cron
A local-first recurring schedule engine for Openclaw Skills that converts natural language timing into structured execution plans.
Prospector
Prospector is an automated lead generation engine that identifies potential client companies based on keywords and extracts verified contact emails from web sources.
Key Tracker: Intelligent Information Capture
Key Tracker is an intelligent information capture skill that acts as an external brain by automatically extracting deadlines, decisions, and tasks from AI interactions.
SkillForge
SkillForge is a specialized generator that builds high-quality, production-grade AI agent skills through a structured 7-step optimization pipeline.
DingTalk OpenClaw Connector
A bridge connecting DingTalk Stream mode to OpenClaw AI for creating intelligent enterprise chatbots.
Repo Visibility Guard
A specialized security hook that prevents GitHub repositories from going public unless a corresponding private metadata repository exists.
Featured*
