An advanced web scraping engine for Openclaw Skills featuring anti-bot bypass, JavaScript rendering, and self-healing adaptive selectors.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install openclaw-scrapling
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install openclaw-scrapling using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapling Web Scraping Skill?
Scrapling is a sophisticated data extraction tool designed for the Openclaw Skills ecosystem. It addresses the challenges of modern web scraping by providing built-in stealth capabilities to bypass Cloudflare Turnstile and sophisticated bot detection systems. Whether dealing with static HTML or complex, JavaScript-heavy Single Page Applications (SPAs) built with React or Vue, Scrapling ensures reliable data delivery.
At its core, this skill provides a robust CLI and Python API that allows developers to automate browser interactions, manage authenticated sessions, and utilize adaptive selector technology. This adaptive feature is particularly valuable for Openclaw Skills users, as it allows scrapers to survive website redesigns by learning and re-locating elements based on structural similarity rather than fixed paths.
Scrapling Web Scraping Skill Use Cases
- Extracting data from websites protected by Cloudflare, Turnstile, or advanced fingerprinting.
- Scraping dynamic content from SPAs that require full JavaScript execution and network idle detection.
- Automating authenticated data collection by persisting login sessions across multiple runs.
- Building resilient monitoring tools that adapt to frequent UI changes without manual selector updates.
- Converting web documentation or articles into Markdown for AI training and knowledge bases.
How Scrapling Web Scraping Skill Works
- The skill receives a target URL and extraction parameters through the scrape.py entry point or a custom script.
- It initializes the appropriate fetcher: Basic for speed, Stealthy for bot-protected sites, or Dynamic for JavaScript rendering.
- For protected targets, it spoofs browser fingerprints and handles automated challenges transparently.
- If adaptive mode is enabled, it checks the selector_cache.json to identify previously learned patterns for the target elements.
- The engine executes CSS or XPath selectors to isolate data, supporting pseudo-elements for clean text and attribute extraction.
- The extracted data is serialized into the requested format (JSON, CSV, Markdown, etc.) and saved to the local file system.
Scrapling Web Scraping Skill Setup
To use this within your Openclaw Skills environment, ensure you have Python 3 and pip available. The skill will handle its own browser dependencies upon the first execution.
# Install the core library
pip install scrapling
# Basic usage to verify installation
python scrape.py --url "https://example.com" --selector "h1" --output test.json
Scrapling Web Scraping Skill Data Schema & Taxonomy
Scrapling organizes its operations and outputs using a structured file hierarchy to support persistent Openclaw Skills workflows:
| Component | Type | Description |
|---|---|---|
output |
File | The primary data export in formats like .json, .csv, .md, or .jsonl. |
sessions/ |
Directory | Stores browser state and cookies for authenticated sessions. |
selector_cache.json |
JSON | Stores structural signatures of elements for adaptive self-healing. |
examples/ |
Scripts | Reference Python implementations for stealth, dynamic, and basic modes. |
Scrapling Web Scraping Skill Advanced Features
- StealthySession Management: Maintain persistent logins across different scraping tasks to avoid repeated authentication hurdles.
- Adaptive Selector Engine: Uses similarity algorithms to automatically find moved or renamed elements after a site redesign.
- Multi-Driver Architecture: Toggle between high-speed HTTP fetching and full Playwright-based browser automation.
- Custom Spider Support: Users can write class-based spiders within the Openclaw Skills framework for complex crawling and pagination logic.
- Fingerprint Spoofing: Automatically generates realistic browser headers and hardware profiles to minimize detection risk.
SKILL.md
Loading
Related Openclaw Skills
Unity Level Design
A comprehensive automation toolkit for rapidly prototyping Unity environments and gameplay systems through AI-driven editor scripts.
Game Developer
A specialized AI agent skill designed to architect, implement, and optimize high-performance games using Unity, Unreal Engine, and ECS patterns.
Fuku Sportsbook Agent Skill
An automated sports betting and data analysis skill for creating AI agents that track odds, post picks, and manage virtual or real USDC bankrolls.
fuku-predictions
A conversational AI trading agent that uses Fuku sports models to scan and execute trades on Kalshi prediction markets based on natural language preferences.
ArbInjection
A real-time security scanner that monitors EVM blockchains to detect dangerous arbitrary call injection vulnerabilities in smart contracts.
Kamino Positions Monitor
A professional-grade monitor for Solana wallets that tracks Kamino Lend positions, provides liquidation risk reports, and suggests actionable safety measures.
Featured*
