A professional-grade web extraction tool that transforms any website URL into structured Markdown content using the BrowserAct API.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install web-search-scraper-api-skill
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install web-search-scraper-api-skill using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Web Search Scraper API Skill?
The Web Search Scraper API Skill is a robust integration designed for developers and AI agents who need to bridge the gap between raw HTML and clean, usable Markdown. By leveraging the BrowserAct Web Search Scraper API, this skill bypasses common scraping hurdles like reCAPTCHA, IP geofencing, and complex JavaScript rendering. It provides a reliable stream of structured data without the hallucinations often associated with pure LLM-based scraping.
Built to be a core part of the Openclaw Skills ecosystem, this tool prioritizes speed and cost-effectiveness. Instead of consuming excessive tokens to have an AI interpret a webpage, this skill uses a specialized workflow to deliver the direct text content of articles, blogs, and documentation in a format that is immediately ready for documentation engines, LLM context windows, or static site generators.
Web Search Scraper API Skill Use Cases
- Extracting full article content from news sites and blogs into clean Markdown.
- Scraping technical documentation from software websites for offline local knowledge bases.
- Converting complex HTML landing pages into readable text for data analysis.
- Bypassing anti-scraping measures like reCAPTCHA during automated data collection.
- Automating the ingestion of web-based tutorials into personal knowledge management systems.
How Web Search Scraper API Skill Works
- The agent receives a target URL and checks for a valid BROWSERACT_API_KEY environment variable.
- A Python-based execution script is triggered, passing the target URL as a parameter to the BrowserAct API.
- The system monitors real-time status logs with timestamps to track the automated browser operation.
- The BrowserAct engine handles rendering, bypasses verification challenges, and extracts the core content.
- The skill receives the raw response, parses the data, and outputs the final Markdown content to the user or agent environment.
Web Search Scraper API Skill Setup
To use this Openclaw Skills integration, ensure you have Python installed and your API credentials ready.
- Obtain your API Key from the BrowserAct Console.
- Set your environment variable:
export BROWSERACT_API_KEY='your_api_key_here'
- Run the scraper via the CLI:
python -u ./scripts/web_search_scraper_api.py "https://example.com"
Web Search Scraper API Skill Data Schema & Taxonomy
The skill returns a structured response focusing on content fidelity. Below is the output structure:
| Field | Type | Description |
|---|---|---|
| content | string | The complete, parsed Markdown representation of the webpage. |
| status_logs | string | Real-time execution logs (e.g., Task Status: running). |
All metadata and headers are automatically handled to ensure the output is purely the requested Markdown content.
Web Search Scraper API Skill Advanced Features
- Anti-Verification Bypass: Automatically handles reCAPTCHA and other human-machine verification challenges.
- Geofencing Resilience: Bypasses IP-based access restrictions and regional blocks.
- Smart Retry Logic: Automatically attempts a second execution if the first fails due to network fluctuations.
- Real-Time Monitoring: Continuous terminal output allows agents to distinguish between long-running tasks and deadlocks.
- Low-Latency Extraction: Optimized for speed compared to purely generative AI browser automation.
SKILL.md
Loading
Related Openclaw Skills
Spc Reference Tool
A zero-latency, local reference utility that provides instant access to technical documentation, patterns, and security best practices.
Hillstone Threat Intelligence Skill
A high-performance security skill for querying IP, domain, URL, and file hash reputation data from the Hillstone threat intelligence database.
MCSManager Controller
A professional-grade controller for MCSManager-managed Minecraft instances, enabling seamless API-driven server operations, log tailing, and troubleshooting.
Polymarket Prediction Tracker
A comprehensive prediction market toolset for querying real-time odds, tracking price momentum, and simulating trades on Polymarket.
Refuel Record: Daxin Fleet Gas Log Automation
A specialized automation tool for Daxin Fleet to process raw refueling data into comprehensive gas summary sheets and card deduction logs.
Memory (Memoria)
A high-performance durable memory system for OpenClaw agents to store, retrieve, and manage long-term project context and user preferences.
Featured*
