A high-performance web scraping tool designed to bypass anti-bot protections and convert web content into clean Markdown for AI agents.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapling-fetch-basic
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapling-fetch-basic using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapling Fetch Basic?
Scrapling Fetch Basic is a robust entry-level tool within the Openclaw Skills ecosystem designed for efficient web data extraction. It specializes in capturing content from static pages such as blogs, news sites, and public announcements. By integrating advanced smart extraction logic, it identifies the core article text without requiring manual CSS selector configuration.
The skill provides a seamless bridge between raw web data and AI-ready information by automatically converting complex HTML into clean, structured Markdown. Whether you are dealing with standard websites or pages protected by Cloudflare, this tool ensures high-quality data retrieval through its versatile operating modes.
Scrapling Fetch Basic Use Cases
- Scraping blog posts and news articles for knowledge base population
- Extracting technical documentation or public announcements for AI analysis
- Bypassing basic anti-bot systems like Cloudflare Turnstile during research
- Automated content migration from legacy websites to Markdown-based platforms
How Scrapling Fetch Basic Works
- The user provides a target URL and chooses between basic (fast HTTP) or stealth (headless browser) mode.
- In stealth mode, the skill utilizes Playwright with browser fingerprinting to mimic human behavior and bypass protection layers.
- The system executes a prioritized waterfall of 11 smart selectors to accurately locate the main content body.
- Extracted HTML is processed through the html2text engine to strip noise and format the text into Markdown.
- The final output is delivered as either a raw string or a structured JSON object depending on the user's preference.
Scrapling Fetch Basic Setup
To integrate this into your workflow, ensure you have the necessary environment set up for Openclaw Skills. Install the core dependencies and use the following commands:
# Basic fetch for static pages
python3 scripts/scrapling_fetch.py https://example.com/articlenn# Fetch with stealth mode for protected sites
python3 scripts/scrapling_fetch.py https://protected-site.com --mode stealth
# Output results in JSON format for automated pipelines
python3 scripts/scrapling_fetch.py https://example.com --json
Scrapling Fetch Basic Data Schema & Taxonomy
The skill organizes data by prioritizing structural elements to ensure content integrity. Below is the schema for the extraction logic:
| Feature | Description |
|---|---|
| Output Format | Markdown (default) or JSON |
| Character Limit | Default 30,000 characters (customizable) |
| Selector Hierarchy | 11-step fallback (article > main > .post-content > body) |
| Engine | scrapling core with html2text transformation |
Scrapling Fetch Basic Advanced Features
- Stealth Mode Integration: Uses headless browser automation to navigate sites with Cloudflare Turnstile and other bot detection.
- Smart Content Detection: Automatically scans for 11 different content signatures (like
.entry-contentor[class*='body']) to find the signal in the noise. - Flexible Output Control: Supports custom character limits to optimize token usage for LLM processing.
- Openclaw Skills Compatibility: Built to work natively within the Openclaw ecosystem for multi-agent research workflows.
SKILL.md
Loading
Related Openclaw Skills
Regulation Extractor
A professional-grade pipeline for extracting, cleaning, and synchronizing structured regulatory clauses from complex construction engineering PDFs to cloud databases.
Rules of the Claw
A battle-tested library of 56 deterministic regex rules for OpenClaw Guardian that blocks high-risk agent actions at the tool layer without LLM overhead.
Ecommerce Review Analyzer
An automated sentiment analysis tool that scrapes ecommerce reviews and generates professional PDF/Word reports using AI.
PDF to PPT Conversion Skill
A specialized conversion tool that transforms visual-rich PDF files into PowerPoint presentations by rendering pages as high-quality images.
Financial Report Analyzer
A sophisticated AI agent skill that extracts key financial indicators from PDF reports and generates comprehensive visual analysis and investment insights.
yacli Yandex
A comprehensive integration for managing Yandex Mail, Disk, and Calendar via the yacli MCP server.
Featured*
