An intelligent data extraction tool designed to crawl websites and APIs to generate structured data in JSON, CSV, or Excel formats.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install ecommerce-scraper-pro
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install ecommerce-scraper-pro using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Data Scraper?
Data Scraper is a robust automation tool within the Openclaw Skills ecosystem that simplifies the process of gathering information from the web. By utilizing advanced libraries like Playwright and BeautifulSoup, it allows developers to handle complex JavaScript-rendered pages and extract specific datasets—such as product listings, news articles, or job postings—with high precision.
This skill bridges the gap between raw web content and actionable structured data, ensuring seamless integration into larger AI-driven pipelines. Whether you are dealing with REST APIs or deep-web crawling, Data Scraper provides the intelligent request control and cleaning features necessary for professional-grade data acquisition.
Data Scraper Use Cases
- Monitoring e-commerce prices and inventory across multiple platforms to stay competitive.
- Aggregating job listings from recruitment portals for labor market research.
- Harvesting news articles or blog posts for sentiment analysis and content curation.
- Extracting real estate data for property value estimation and trend tracking.
- Collecting social media engagement metrics to analyze brand performance.
How Data Scraper Works
- The user provides a target URL, an API endpoint, or a file containing multiple URLs to the scraper.
- The engine identifies the data structure using pre-defined types (like product or article) or custom CSS/XPath selectors.
- If required, the system renders dynamic content through a headless browser to ensure all JavaScript-loaded data is visible.
- Structured data is extracted, cleaned through automatic deduplication, and formatted according to user preferences.
- The results are saved to a local file or passed to other Openclaw Skills for downstream processing.
Data Scraper Setup
To begin using this skill within your environment, use the following commands:
# Scrape a single webpage with a specific selector
uv run scripts/data-scraper.py scrape --url "https://example.com/products" --selector ".product"
# Process multiple URLs from a text file
uv run scripts/data-scraper.py scrape --urls-file urls.txt --output data.json
# Fetch data from a secure API
uv run scripts/data-scraper.py api --endpoint "https://api.example.com/data" --auth "Bearer TOKEN"
Data Scraper Data Schema & Taxonomy
The Data Scraper organizes its output into a consistent format to ensure compatibility with other Openclaw Skills.
| Field | Description | Example |
|---|---|---|
url |
The source address of the scraped data | https://example.com |
scrapedAt |
ISO timestamp of the extraction event | 2026-02-28T01:13:00Z |
data |
Array containing extracted objects | [{ "title": "Product", "price": "$99" }] |
count |
Total number of items successfully extracted | 50 |
format |
The output file type (json, csv, or xlsx) | json |
Data Scraper Advanced Features
- Multi-format output support including JSON, CSV, and formatted Excel workbooks with pivot table capabilities.
- Anti-scraping evasion using intelligent request frequency control and proxy rotation.
- Scheduled task execution using cron syntax for automated periodic data harvesting.
- Custom field definition using CSS or XPath for sites that do not match standard data templates.
- Seamless integration with REST and GraphQL APIs for unified data management.
SKILL.md
Loading
Related Openclaw Skills
IMDb API
Integrate 19 specialized IMDb operations into your AI agent for comprehensive title research, cast analysis, and release monitoring.
Telegram Agent Setup
A definitive guide and technical workflow for integrating OpenClaw agents with the Telegram messaging platform for secure, automated communication.
JD.com API
A specialized tool for extracting precise JD.com product details, customer reviews, and shop inventories via the JustOneAPI interface.
ClawGuard Guardian
ClawGuard Guardian is a runtime security monitor that intercepts, controls, and replays AI agent operations to ensure safe and compliant behavior.
ClawGuard Threat Detector (CG-TD)
An enterprise-grade runtime security system for OpenClaw environments that detects behavioral anomalies and multi-stage attacks in real-time.
Logistics & Customs Optimizer
An expert AI agent designed to optimize shipping routes, calculate customs duties, and manage VAT compliance for cross-border e-commerce in the EU market.
Featured*
