A powerful web content extraction tool for Openclaw Skills that converts URLs into clean markdown, HTML, or JSON using the Bright Data CLI.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install brightdata-scrape
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install brightdata-scrape using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Bright Data Scrape?
The Bright Data Scrape skill is a robust solution designed for AI agents within the Openclaw Skills ecosystem to acquire raw or lightly-structured web content. It allows developers and agents to fetch page data in multiple formats, including LLM-friendly markdown, full HTML, or structured JSON. This skill is particularly valuable when you need to bypass complex bot detection and retrieve accurate data for further analysis or RAG workflows.
By integrating directly with the Bright Data CLI, this skill provides a reliable bridge between the open web and your AI's knowledge base. It handles the nuances of geo-targeting and output formatting, ensuring that Openclaw Skills can access the precise information they need from any global source.
Bright Data Scrape Use Cases
- Extracting clean article content into markdown for summarization or knowledge indexing.
- Capturing raw HTML for deep DOM analysis or metadata harvesting.
- Scraping paginated lists of links to build comprehensive datasets for research.
- Generating PNG screenshots of web pages for visual verification or UI monitoring.
- Bypassing geographic restrictions by routing requests through specific country exit nodes.
How Bright Data Scrape Works
- The agent identifies the target URL and verifies the local environment has the Bright Data CLI configured for Openclaw Skills.
- A setup gate check is performed to ensure the bdata tool is authenticated and active.
- The skill determines the most efficient path, choosing between single-page fetches or parallelized batch processing.
- The scrape command is executed with specific format flags such as markdown, html, or json.
- A verification gate inspects the output for quality, ensuring no block pages or CAPTCHAs were encountered.
- The final cleaned content is delivered to the agent or saved to the specified output path.
Bright Data Scrape Setup
To utilize this skill within the Openclaw Skills framework, ensure the Bright Data CLI is installed and your session is authenticated.
# Install the Bright Data CLI via npm
npm install -g @brightdata/bdata-cli
# Log in to your Bright Data account
bdata login
# Verify connection to active zones
bdata zones
Bright Data Scrape Data Schema & Taxonomy
The skill organizes its output based on the requested format, providing clear data structures for Openclaw Skills to process:
| Format | Extension | Description |
|---|---|---|
| Markdown | .md | Cleaned text content with headers and links preserved for AI processing. |
| HTML | .html | The full raw DOM source of the target webpage. |
| JSON | .json | Structured data fields returned when using the Web Unlocker feature. |
| Screenshot | .png | A visual capture of the page as rendered in a browser. |
Bright Data Scrape Advanced Features
- Parallelized batch scraping using xargs to handle large lists of URLs without performance degradation.
- Automatic geo-targeting overrides to access localized content from specific countries.
- Smart error escalation that transitions from simple scraping to full browser rendering when encountering heavy JavaScript.
- Integrated block-page detection signatures to maintain high success rates for Openclaw Skills tasks.
- Seamless hand-off logic to specialized data feeds for platforms like LinkedIn, Amazon, and TikTok.
SKILL.md
Loading
Related Openclaw Skills
Hivebrite AI Skill
A powerful integration for AI agents to programmatically manage Hivebrite community data, member records, and automated engagement workflows.
HR Cloud
An AI-powered integration for the HR Cloud platform that enables automated management of employee records, time-off requests, and recruitment workflows.
Hotmart
A comprehensive integration for managing Hotmart digital products, sales transactions, and affiliate marketing campaigns via AI agents.
Infinity Integration for AI Agents
Seamlessly integrate Infinity project management into your AI workflows to manage boards, tasks, and team collaboration via Openclaw Skills.
Hostaway
A professional integration skill for property managers to automate Hostaway vacation rental operations, bookings, and guest communications.
SEO Audit (Bright Data)
A high-fidelity SEO diagnostic tool that leverages the Bright Data CLI to identify technical issues and ranking opportunities using live, JS-rendered web data.
Featured*
