A Dockerized browser-backed scraping service that fetches HTML from JavaScript-heavy or Cloudflare-protected websites via an HTTP API.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapclaw
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapclaw using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapclaw?
Scrapclaw is a specialized tool designed to bridge the gap between simple HTTP requests and the complex requirements of modern, dynamic web pages. As part of the Openclaw Skills ecosystem, it provides a Dockerized browser instance that renders JavaScript-heavy content and navigates through Cloudflare protection before returning the data to the agent. This approach ensures that technical hurdles like client-side rendering do not prevent data extraction.
By running as a self-hosted service, it offers developers a private and scalable way to integrate robust web crawling capabilities into their workflows. Whether you are running it on a local machine or a remote server, this skill translates raw web data into actionable information for AI agents, allowing them to see the web exactly as a human user would.
Scrapclaw Use Cases
- Scraping content from Single Page Applications (SPAs) like React, Angular, or Vue sites.
- Bypassing Cloudflare anti-bot challenges to access public data for research.
- Extracting clean, readable text from complex layouts for LLM processing.
- Automating data collection from pages that require a post-navigation wait for dynamic elements to load.
How Scrapclaw Works
- The skill first identifies the target URL and checks if the Scrapclaw Docker service is reachable via a health check.
- A POST request is sent to the service's /v1 endpoint containing configuration parameters like timeout and wait times.
- The Scrapclaw container launches a browser instance to navigate to the target site, executing all necessary JavaScript.
- Once the page is rendered or the specified wait time elapses, the service captures the HTML or extracts the text.
- The skill receives the processed response and provides it to the agent, ensuring that the fetched content is treated as untrusted input for safety.
Scrapclaw Setup
Run the published Docker image from the GitHub Container Registry:
docker run --rm -d \
--name scrapclaw \
-p 8192:8192 \
ghcr.io/ericpearson/scrapclaw:v0.0.6
Install the skill into your local Openclaw Skills directory:
mkdir -p ~/.openclaw/workspace/skills
cp -R skills/scrapclaw ~/.openclaw/workspace/skills/
Alternatively, install directly via ClawHub:
clawhub install scrapclaw --version 0.0.6
Scrapclaw Data Schema & Taxonomy
The skill interacts with the Scrapclaw API using the following JSON schema for requests and responses:
| Field | Description | Type |
|---|---|---|
| url | The target URL to be scraped (Required) | String |
| maxTimeout | Maximum duration for the request in ms (Default: 60000) | Integer |
| wait | Extra delay after page load in ms (Default: 0) | Integer |
| responseMode | Choose between 'html' or 'text' output | String |
| maxResponseBytes | Optional cap for the UTF-8 byte response | Integer |
Success responses return a JSON object containing solution.response, solution.status, and solution.title.
Scrapclaw Advanced Features
- Cloudflare challenge solving to access protected web resources.
- Configurable
waitparameter to ensure dynamic components are fully rendered before capture. - Dual
responseModesupport to switch between raw markup for structure and text for content analysis. - Sensitive data handling through
SCRAPCLAW_API_TOKENfor secure, authorized scraping. - Isolated execution environment using Docker to protect the host system from untrusted web content.
SKILL.md
Loading
Related Openclaw Skills
Personal Insight Engine (PIE)
A strategic analysis tool that distills actionable insights from local session logs using advanced LLMs.
Bracket Oracle
A sophisticated March Madness optimization tool that utilizes Monte Carlo simulations and statistical efficiency data to generate winning tournament brackets.
Smart Commit
Smart Commit is an AI-powered git utility that analyzes staged changes to generate precise, structured commit messages following the Conventional Commits standard.
Tortoise Virtual Pet Skill
A sophisticated virtual pet management skill for AI agents featuring a 90-day growth marathon and stoic metabolic mechanics.
Tabby Virtual AI Cat
A virtual pet care skill for AI agents to adopt, feed, and raise a social Tabby cat via real-time API interactions and emotional feedback loops.
Memory Tasks
A structured task management system for AI agents that ensures complex workflows survive context loss through persistent knowledge graph notes.
Featured*
