An intelligent data extraction utility that converts web pages and API responses into clean, structured datasets like JSON, CSV, and Excel.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install ecommerce-data-scraper-pro
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install ecommerce-data-scraper-pro using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Data Scraper?
The Data Scraper skill is a robust solution for developers needing to harvest high-quality data from the web or external services. By integrating this tool into your AI agent environment, you can automate the process of navigating complex websites, handling JavaScript rendering, and extracting specific information without manual intervention. It serves as a bridge between unstructured web content and actionable data, specifically optimized for use within the Openclaw Skills ecosystem.
Built with a focus on reliability and flexibility, the skill handles everything from simple single-page scrapes to complex, multi-threaded batch operations. It includes built-in intelligence to manage request frequencies and bypass common anti-scraping measures, ensuring your data pipelines remain stable and efficient.
Data Scraper Use Cases
- Tracking e-commerce pricing and inventory across multiple retail platforms.
- Aggregating job market data and salary information for competitive analysis.
- Collecting news content and blog articles for automated research and summarization.
- Monitoring social media metrics including post engagement and comments.
- Fetching and formatting data from REST or GraphQL APIs for internal reporting.
How Data Scraper Works
- The user initiates a request by providing a target URL, a file of URLs, or an API endpoint.
- The skill determines the optimal parsing method, using Playwright for JavaScript-heavy sites or BeautifulSoup for static content.
- Data is identified using either pre-defined schemas (like product or article) or custom CSS/XPath selectors.
- Intelligent rate limiting and proxy management are applied to maintain access and follow robots.txt guidelines.
- The tool cleans and de-duplicates the retrieved data before formatting it into the requested output type.
Data Scraper Setup
To deploy this skill within your Openclaw Skills setup, use the following commands to initialize the scraper:
# Basic scraping of a single URL
uv run scripts/data-scraper.py scrape --url "https://example.com/products" --selector ".product"
# Batch scraping from a list of URLs
uv run scripts/data-scraper.py scrape --urls-file urls.txt --output results.json
Ensure your environment variables are configured if using proxies or specific API authentication tokens.
Data Scraper Data Schema & Taxonomy
The skill organizes extracted data into standardized schemas to ensure compatibility with downstream analytics tools. Below are the primary data models supported:
| Type | Description | Key Fields |
|---|---|---|
| product | E-commerce items | price, name, rating, stock_status |
| article | Editorial content | title, author, published_date, body |
| job | Career listings | position, company, location, salary_range |
| social | Social platforms | post_text, user, likes, timestamp |
All outputs include metadata such as the source URL and the timestamp of the scrape for auditability.
Data Scraper Advanced Features
- Multi-format export support including JSON, CSV, and fully formatted Excel workbooks.
- Integrated scheduling capabilities using cron syntax for automated, periodic data harvesting.
- Advanced anti-bot protection featuring intelligent delays and proxy rotation.
- Support for complex API interactions including Bearer token authentication and GraphQL queries.
- Custom field definition allowing users to map any DOM element to a structured data key via CSS/XPath.
- High-performance batch processing for large-scale data collection tasks within Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
CartPilot
CartPilot is a strategic checkout decision layer that optimizes order paths across major Chinese e-commerce platforms to balance price, speed, and effort.
Molty Royale Agent Operation
A comprehensive skill for operating autonomous AI agents within the Molty Royale blockchain gaming ecosystem.
Solana Monitor
A comprehensive real-time monitoring suite for tracking Solana ecosystem assets, prices, and high-value transactions.
AI Copywriter
A professional-grade AI marketing assistant that generates high-conversion copy for ads, social media, and e-commerce platforms.
AI Email Master
AI Email Master is a professional email composition assistant that provides tone adjustment, template libraries, and grammar checking for high-impact communication.
AI Social Pro
AI Social Pro is a comprehensive social media management platform that leverages AI for content creation, scheduling, and performance analytics across major global and Chinese platforms.
Featured*
