An automated discovery and acquisition utility for finding datasets, public data lakes, and open-data portals across multiple global repositories.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install dataset-search
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install dataset-search using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Dataset Search?
The dataset-search skill is a robust data discovery and retrieval framework designed to locate, compare, and obtain datasets for machine learning, business intelligence, climate analysis, NLP, and more. It allows developers and AI agents to systematically query public repositories, cloud data registries, and open-data portals through a unified standard-library Python command-line utility.
As part of the Openclaw Skills ecosystem, this tool handles the heavy lifting of dataset sourcing, license verification, and structured comparison. Rather than manually hunting for high-quality benchmarks or open-data portals, users can run scoped profile searches and generate actionable data acquisition plans automatically.
Dataset Search Use Cases
- Sourcing training, validation, and benchmark datasets for AI models and NLP tasks.
- Harvesting public environmental, climate, and geospatial data for spatial analysis and forecasting.
- Searching for econometric, government, and financial data registries to support business intelligence and analytics.
- Finding high-quality, pre-labeled multi-modal data for RAG and vector database ingestion projects.
- Safely planning and executing bulk data downloads while respecting license and storage restrictions.
How Dataset Search Works
- Define the Data Brief: Analyze the user's analytic goal and convert it into a structured target profile including domain, task, geography, language, and file format constraints.
- Broad Multi-Repository Querying: Execute the bundled script across broad source catalogs or specific domains (e.g., Hugging Face, Kaggle, OpenML) to find candidates matching the search criteria.
- Scope and Source Refinement: Filter queries using targeted profiles such as geospatial, climate, or economics to query niche providers (e.g., Copernicus, AWS Open Data, FRED).
- Evaluate and Compare: Inspect found dataset candidates based on relevance, update cadence, metadata quality, size, license, and direct access compatibility.
- Execution and Retrieval: Construct a safe data acquisition plan. For standard datasets, execute a local download. For massive cloud lakes, generate optimal SQL or S3 direct querying scripts instead of local downloading.
Dataset Search Setup
Requirements
- Python 3.x
CLI Execution
To utilize this entry from the Openclaw Skills library, run the provided Python script directly from your terminal. No third-party dependencies are strictly required for basic searches, as the script uses standard Python libraries.
To perform a quick dataset search:
python3 skills/dataset-search/scripts/dataset_search.py search "global climate temperature" --profile climate --limit 5
To list all available dataset sources:
python3 skills/dataset-search/scripts/dataset_search.py sources
Setup for Source-Specific CLIs (Optional but Recommended)
For full downloading capabilities, ensure you have the respective client utilities configured in your environment:
- Kaggle: Install the kaggle CLI tool and place your API credentials in
~/.kaggle/kaggle.json. - Hugging Face: Install
huggingface-cliand runhuggingface-cli loginfor authenticated access.
Dataset Search Data Schema & Taxonomy
The dataset-search tool organizes search parameters, target briefs, and download outputs in structured environments. Highly modular search profiles align query keywords with target sectors.
Available Search Profiles
| Profile | Intended Domain | Common Sources Covered |
|---|---|---|
ml / general |
Machine Learning, AI | Kaggle, OpenML, Hugging Face, UCI, Zenodo |
nlp |
Natural Language Processing | Hugging Face, Common Crawl, The Pile |
geospatial / climate |
Remote sensing, Weather | AWS, Copernicus, NASA Earthdata, NOAA |
economics |
Finance, Econometrics | FRED, IMF, World Bank, UN Data |
government |
Public sector statistics | data.gov, data.europa.eu, dados.gov.br |
Output Schemas
When running searches with --format json --output /path/to/results.json, the generated file yields an array of dataset objects with the following metadata structure:
- title: The human-readable name of the dataset.
- source: Origin repository or platform ID (e.g.,
kaggle,huggingface). - url: Permanent link to the source repository web page.
- license: Verified licensing status if exposed via API.
- access_method: Indicator whether the dataset supports direct download, CLI retrieval, or cloud-native querying.
Dataset Search Advanced Features
- Structured Dataset Briefs: Utilize the
--briefflag with a structured JSON parameters file defining precise task boundaries, periods, scale, and licenses to automate advanced discovery workflows. - Safe Acquisition Planning: The download utility acts defensively. Running
downloadwithout the--yesflag generates a detailed, readable preview plan outlining expected sizes and credential requisites before fetching bytes. - Offline Fallback Protocol: Pass the
--offlineflag to run searches without triggering network calls, returning reliable fallback search URLs, query parameters, and portal deep links instantly. - Cloud-Native Native Access Routing: Prevents bandwidth bottlenecks on large-scale databases by identifying and supplying S3 pathing, Athena schemas, or BigQuery targets instead of attempting a local disk download.
- Multi-Source Adapter Extensibility: Easily add custom CKAN, Socrata, or direct API routes within the Python framework to align with private enterprise registries or team repositories.
SKILL.md
Loading
Related Openclaw Skills
Doctor Call
An automated diagnostic and repair tool designed to maintain high availability for the OpenClaw gateway using systemd auto-restarts.
Google Messages Browser Skill
A browser automation skill that enables AI agents to send, receive, and monitor SMS/RCS messages directly through Google Messages for Web.
freemoney
An AI-powered tracking skill for OpenClaw that monitors global class-action lawsuit settlements and identifies no-proof claims in Chinese.
RSS Sitemap Discovery
An advanced site discovery skill that maps website structures, RSS feeds, and Atom feeds to optimize AI web crawling and content extraction.
The Psychology of Selling
An AI-powered sales coaching agent that brings Brian Tracy's iconic sales methodology directly to your development and communication workflows.
Codex Session Image Scrubber
A deterministic command-line utility designed to clean oversized Codex session logs by replacing heavy base64 image payloads with small placeholders while retaining full conversation history.
Featured*
