A lightweight web collection utility for extracting structured data and monitoring site changes without a headless browser.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install data-scraper
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install data-scraper using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Data Scraper?
Data Scraper is a powerful utility designed for efficient web data collection and structured text extraction. By utilizing curl-based parsing instead of heavy browser environments, it provides a fast and resource-efficient way to transform raw HTML into usable formats. As part of the Openclaw Skills ecosystem, this tool simplifies complex scraping tasks into straightforward CLI commands, making it ideal for developers and data analysts who need speed and reliability.
The tool excels at stripping away unnecessary boilerplate like scripts and styles, leaving behind clean, readable content. Whether you are performing market research or building a local knowledge base, Data Scraper provides the surgical precision needed to isolate specific data points from any public URL.
Data Scraper Use Cases
- Extracting clean markdown text from blog posts, news articles, and technical documentation.
- Scraping product listings and pricing data for e-commerce competitive analysis.
- Monitoring specific web elements to track price drops or inventory changes.
- Automating batch data collection from a list of URLs for large-scale research.
- Converting legacy HTML tables into modern, structured JSON or CSV formats.
How Data Scraper Works
- The user initiates a request via the CLI using specific modes like fetch, extract, or table.
- The skill fetches the target URL's HTML content using optimized curl requests.
- The engine parses the HTML based on provided CSS selectors or default readability logic.
- Extracted data is structured into the user's preferred format, such as JSON, CSV, or Markdown.
- For monitoring tasks, the system compares current snapshots against stored history in the snapshots directory to detect differences.
Data Scraper Setup
To begin using this skill within your environment, ensure you have the necessary permissions for network access. You can test the installation with the following commands:
# Fetch and clean text from a URL
data-scraper fetch "https://example.com"
# Extract specific data using CSS selectors
data-scraper extract "https://example.com" --selector ".price"
Data Scraper Data Schema & Taxonomy
The skill manages data through a structured hierarchy to ensure portability and clarity:
| Component | Type | Description |
|---|---|---|
| Snapshots | File | Timestamped versions of pages stored in data-scraper/snapshots/ |
| Table Output | JSON | An array of objects where keys are derived from table headers |
| Batch Config | TXT | A simple list of URLs for sequential processing |
| Metadata | Object | Includes User-Agent strings, cookie data, and custom headers |
Data Scraper Advanced Features
- Batch scraping with configurable delays and rate limiting to prevent IP blocking.
- Automated change detection and diffing with integration into notification-hub for alerts.
- Full support for authenticated sessions using custom headers and session cookies.
- Ethical scraping modes that respect robots.txt and implement exponential backoff on 429 errors.
- Seamless workflow integration with other Openclaw Skills like web-claude and seo-audit for comprehensive data pipelines.
SKILL.md
Loading
Related Openclaw Skills
Daily Sales Digest
An automated sales aggregation and reporting skill for e-commerce platforms with multi-channel delivery.
Card News Pipeline
An automated workflow for creating and preparing 5-slide Instagram-ready card news image sets from a single topic.
Business Planner
A professional AI agent skill for generating government funding plans, investor pitch decks, and technical infrastructure architecture.
Brand Voice Manager
A specialized tool for defining, managing, and applying consistent brand tones and writing styles across multiple AI-driven communication channels.
Decision Log
A professional decision-tracking skill that records rationales and automatically evaluates results after 30 days.
git-auto
A comprehensive Git automation skill that streamlines status checks, smart commits, and safe pushes with intelligent defaults.
Featured*
