A comprehensive web data extraction tool for OpenClaw that enables multi-threaded crawling, search engine integration, and structured scraping.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install anycrawl
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install anycrawl using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is AnyCrawl Skill?
The AnyCrawl Skill is a high-performance integration designed to empower Openclaw Skills with robust web navigation and data extraction capabilities. It serves as a bridge to the AnyCrawl API, allowing AI agents to navigate complex web environments, bypass JavaScript rendering challenges using Playwright or Puppeteer, and convert raw HTML into clean, LLM-ready formats like Markdown and structured JSON.
By incorporating this skill, developers can automate the gathering of real-time web information, perform deep site crawls, and execute sophisticated search queries. Whether you need to monitor competitors, build a local knowledge base, or extract specific data points using custom schemas, this skill provides the necessary infrastructure for reliable web-scale automation within the OpenClaw ecosystem.
AnyCrawl Skill Use Cases
- Perform automated market research by searching Google and scraping top results for competitive analysis.
- Build and update RAG (Retrieval-Augmented Generation) datasets by crawling entire documentation websites.
- Extract structured e-commerce data like prices and descriptions directly into JSON format using predefined schemas.
- Capture visual snapshots of web pages alongside text content for UI/UX auditing or archival purposes.
- Automate news gathering by scraping specific paths or tags from media outlets and blogs.
How AnyCrawl Skill Works
- The AI agent initiates a request through a function call like anycrawl_scrape or anycrawl_search.
- The skill validates the request and communicates with the AnyCrawl API using your configured API key.
- Depending on the site complexity, the skill utilizes either Cheerio for speed or browser-based engines (Playwright/Puppeteer) for JavaScript rendering.
- For crawling tasks, an asynchronous job is spawned, allowing the agent to track status and retrieve paginated results as they become available.
- The raw web data is processed according to specified filters (tags, paths, or schemas) and returned as structured output.
AnyCrawl Skill Setup
To use this integration within Openclaw Skills, you must first obtain an API key from anycrawl.dev. You can configure the skill using one of the following methods:
Method 1: Environment Variable
export ANYCRAWL_API_KEY="your-api-key"
Method 2: OpenClaw Configuration
openclaw config.patch --set ANYCRAWL_API_KEY="your-api-key"
AnyCrawl Skill Data Schema & Taxonomy
The skill returns standardized responses to ensure compatibility across different Openclaw Skills workflows. Data is organized as follows:
| Property | Type | Description |
|---|---|---|
success |
boolean | Indicates if the operation was successful. |
data |
object/array | Contains the scraped content, search results, or crawl metadata. |
message |
string | Provides additional context or error details. |
formats |
array | Supports output in markdown, html, text, json, and screenshot. |
AnyCrawl Skill Advanced Features
- Multi-engine support: Choose between Cheerio (static), Playwright (SPA), and Puppeteer (metrics) based on your performance needs.
- Structured JSON extraction: Use JSON schemas and user prompts to force the scraper to return specific data fields.
- Asynchronous Crawling: Manage large-scale site crawls with job status tracking, cancellation, and paginated result retrieval.
- Path-based Filtering: Target specific content by including or excluding URL patterns (e.g., /blog/*) during a crawl.
- Search & Scrape Helper: A streamlined function to search Google and immediately scrape the top results in a single step.
SKILL.md
Loading
Related Openclaw Skills
eonik Ad Budget Leak Agent
A specialized AI agent skill that audits Meta Ad accounts to stop budget waste and surface scaling opportunities automatically.
Watermark Remover
A specialized AI skill designed to detect and remove text, image, and pattern watermarks from PDF documents while maintaining file integrity.
Dialogue and Selection Confirmation
An interactive choice and confirmation interface for AI agents to facilitate rapid decision-making and task validation.
Shadow AI Monitor
An enterprise-grade monitoring dashboard for tracking employee AI tool usage and PIPEDA compliance risks.
Google Docs from Markdown
A bridge utility that converts Markdown files into fully editable Google Docs by automating the DOCX conversion and Drive upload workflow.
FB Repost Skill
An automated workflow to scrape Facebook posts, rewrite them with AI, generate custom visuals, and publish to Facebook Pages.
Featured*
