A comprehensive suite of 22 tools for AI agents to perform advanced web scraping, bypass bot detection, and extract structured data from major e-commerce and search platforms.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scraperapi-mcp
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scraperapi-mcp using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is ScraperAPI MCP?
The ScraperAPI MCP skill provides a powerful interface for AI agents to interact with the web without being blocked by sophisticated bot protection. By leveraging Openclaw Skills, developers can integrate high-performance scraping capabilities that include proxy rotation, CAPTCHA bypass, and geo-targeted results.
This skill is designed to handle everything from simple webpage fetching to complex multi-page crawling and structured data extraction from platforms like Google, Amazon, Walmart, eBay, and Redfin. It allows agents to intelligently decide when to use headless rendering or premium proxies, ensuring efficient credit usage while maintaining high success rates for data retrieval.
ScraperAPI MCP Use Cases
- Performing real-time market research and SEO tracking via Google Search and Google News monitoring.
- Automating e-commerce price tracking and product analysis across Amazon, Walmart, and eBay.
- Extracting real estate market data, including property listings and rental prices, from Redfin.
- Bypassing JavaScript-heavy bot protection on SPAs to extract clean Markdown for LLM consumption.
- Orchestrating site-wide crawls to build comprehensive knowledge bases for specific domains.
How ScraperAPI MCP Works
- The AI agent identifies a scraping or research task and selects the most appropriate tool from the 22 available ScraperAPI options.
- The agent configures parameters such as the target URL, geo-location (countryCode), and rendering requirements.
- The skill routes the request through the ScraperAPI MCP server, which manages the lifecycle of the request, including proxy rotation.
- For structured sites, the skill uses Auto-parse to convert raw HTML into clean JSON or Markdown.
- The agent receives the processed data, handling large outputs via incremental reads to maintain context window efficiency.
ScraperAPI MCP Setup
To use this skill, you must have an active ScraperAPI account and a valid API key. Configuration requires setting the environment variables for your MCP server variant.
# Set your ScraperAPI API Key
export SCRAPERAPI_API_KEY='your_api_key_here'
# The skill requires the ScraperAPI MCP server to be running.
# Refer to the specific setup documentation for local vs remote server variants.
ScraperAPI MCP Data Schema & Taxonomy
The skill organizes data based on the source platform and tool type, typically returning content in Markdown or JSON formats.
| Data Category | Tools | Primary Metadata |
|---|---|---|
| General Web | scrape |
url, outputFormat, render, countryCode |
| Search Engines | google_search, google_news |
query, timePeriod, countryCode, start |
| E-commerce | amazon_product, walmart_search |
asin, productId, tld, price, rating |
| Real Estate | redfin_search, redfin_agent |
property_url, listing_status, price_history |
| Crawler | crawler_job_start |
jobId, status, crawlBudget, maxDepth |
ScraperAPI MCP Advanced Features
- Auto-Parsing (SDE): Automatically converts complex HTML from supported sites into structured JSON data for immediate analysis.
- Gradual Escalation Logic: Supports switching between standard, render, premium, and ultra-premium proxies to optimize success versus cost.
- Job Lifecycle Management: Start, monitor, and cancel long-running site crawls with the built-in crawler job tools.
- Localized Research: Utilize geo-targeting and TLD parameters to see the web as a user would in specific global regions.
- Markdown Optimization: Default output format tailored for AI context windows, converting messy HTML into clean, readable Markdown.
SKILL.md
Loading
Related Openclaw Skills
Free Image and Video Generation
A hybrid AI toolkit combining local processing for enhancement and cloud-based generation with over 300 models.
AWS EC2 Management (watadot-aws-ec2)
A comprehensive AWS EC2 orchestration skill for Openclaw Skills that automates cloud compute management and security.
RTK Compressor
An intelligent CLI output compressor designed to reduce LLM token consumption by up to 90% by stripping redundant data.
Defense Lawyer
A comprehensive AI legal assistant for criminal defense professionals, specializing in the Chinese legal system.
Meeting Transcript Action Extractor
An intelligent utility for Openclaw Skills that transforms raw meeting transcripts into structured decisions, action items, and clear summaries.
ValueSider Superinvestor Data
Automates the retrieval and parsing of guru investor portfolio holdings and 13F filings from ValueSider.
Featured*
