A high-performance Python-based web scraping tool designed for resilient data extraction from static, dynamic, and anti-bot protected websites.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapling-extract
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapling-extract using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapling?
Scrapling is a comprehensive web scraping library that empowers AI agents to extract structured data from virtually any website. It distinguishes itself through its multi-modal fetcher system, allowing users to toggle between standard HTTP requests, full browser rendering via Playwright, and stealthy browsers that bypass sophisticated anti-bot measures like Cloudflare. As part of the Openclaw Skills ecosystem, Scrapling provides the technical foundation for robust data pipelines that survive site redesigns thanks to its unique adaptive DOM relocation technology.
By synthesizing CSS selectors, XPath, and session management, Scrapling makes it easy to convert complex web architectures into clean, usable data. Whether you are building an automated crawler or a one-off data extraction script, this skill ensures that your selectors remain stable and your requests stay undetected.
Scrapling Use Cases
- Extracting real-time product pricing and stock information from e-commerce platforms.
- Gathering structured datasets from JavaScript-heavy single-page applications (SPAs).
- Collecting content from websites protected by anti-scraping and browser fingerprinting services.
- Building RAG-compatible datasets by converting web pages directly into Markdown.
- Maintaining long-lived scraping sessions that require cookie persistence and authentication.
How Scrapling Works
- Select the appropriate fetcher mode (Fetcher, Dynamic, or Stealthy) based on the target website's rendering and security complexity.
- Execute the request to retrieve the page content, ensuring that headers, cookies, or browser states are properly managed.
- Parse the DOM using CSS or XPath selectors, with the option to enable adaptive relocation for increased resilience against UI changes.
- Extract the required data points into structured formats like JSON for automation or Markdown for summarization.
- Validate the extracted data against expected schemas and handle errors or retries to ensure high reliability.
Scrapling Setup
Install the base package via pip:
pip install scrapling
For browser-based rendering and anti-bot features, install the fetcher extras:
pip install "scrapling[fetchers]"
scrapling install
python3 -m playwright install
Scrapling Data Schema & Taxonomy
Scrapling organizes data extraction through specialized fetcher classes and output contracts:
| Component | Functionality |
|---|---|
| Fetcher | Optimized for static HTML and API-like responses. |
| DynamicFetcher | Utilizes Playwright for JavaScript rendering. |
| StealthyFetcher | Employs advanced fingerprinting to bypass anti-bot systems. |
| Adaptive Selector | Uses saved DOM snapshots to relocate elements when site structures change. |
| Output Formats | Supports JSON for data pipelines and Markdown for LLM ingestion. |
Scrapling Advanced Features
- Adaptive DOM relocation that automatically updates selectors when site layouts change.
- Stealth browser fingerprinting to successfully crawl Openclaw Skills targets protected by Cloudflare.
- Native Playwright integration for complex client-side interactions and event handling.
- Integrated session management (FetcherSession, StealthySession) for persistent cookie and state handling.
- Command-line interface for rapid, no-code data extraction from any terminal.
SKILL.md
Loading
Related Openclaw Skills
Local Pandoc
A versatile document conversion skill that enables AI agents to transform Markdown files into professional PDFs, HTML, and Word documents using the Pandoc utility.
DHH Rails Style
A specialized AI skill for writing Ruby and Rails code that strictly follows the 37signals and DHH philosophy of clarity and REST purity.
QVeris Tool Discovery & Execution
A dynamic tool discovery and execution engine that connects AI agents to thousands of external APIs via the QVeris API.
Vincent - A Wallet for AI Agents
A policy-controlled smart wallet that enables AI agents to execute blockchain transactions safely without ever exposing private keys.
ModelWar AI CoreWar Arena
A competitive arena where AI agents battle in a virtual computer using the classic CoreWar programming game.
mflux: Local Apple Silicon Image Generation
A high-performance MLX-based skill for generating images locally on Apple Silicon using FLUX.2 and Z-Image models.
Featured*
