A high-performance web scraping skill that leverages ScrapeNinja's proxy and headless browser capabilities to extract structured web data via Membrane.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapeninja
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapeninja using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is ScrapeNinja?
The ScrapeNinja skill enables AI agents to perform complex web scraping tasks without worrying about the underlying infrastructure. By integrating with Openclaw Skills, it provides a streamlined interface for managing proxies, bypassing anti-bot measures, and utilizing headless browsers to retrieve clean, structured data from any website.
Built on the Membrane framework, this skill automates the most difficult parts of data extraction. It handles secure credential management and session persistence, allowing developers to focus on the data logic rather than the plumbing of HTTP requests and proxy rotation.
ScrapeNinja Use Cases
- Extracting product pricing and availability from e-commerce websites at scale.
- Gathering structured data for market research and competitive analysis.
- Monitoring news sites or blogs for specific keywords and updates.
- Bypassing sophisticated anti-scraping technologies using ScrapeNinja's managed proxy network.
- Automating lead generation by scraping public directory information.
How ScrapeNinja Works
- Installation: The Membrane CLI is installed to provide the primary interface for the Openclaw Skills environment.
- Authentication: Users authenticate via a secure tenant-based login that manages tokens server-side.
- Connection Establishment: A connection is ensured to the ScrapeNinja domain, handling the linking of API credentials.
- Action Discovery: The agent searches for specific scraping actions (e.g., 'scrape-page') using natural language intent.
- Execution: The scraping task is sent to ScrapeNinja, which processes the request through its proxy network and returns the structured result to the agent.
ScrapeNinja Setup
First, install the global CLI for Openclaw Skills management:
npm install -g @membranehq/cli@latest
Authenticate your environment (replace <agentType> with your specific agent like openclaw or claude):
membrane login --tenant --clientName=<agentType>
Establish the connection to ScrapeNinja:
membrane connection ensure "https://scrapeninja.net/" --json
ScrapeNinja Data Schema & Taxonomy
The skill organizes data around scraping tasks and connection metadata. Results are typically returned in structured JSON format following the ScrapeNinja API specifications.
| Entity | Description |
|---|---|
| Connection | Stores the authenticated state and API keys for the ScrapeNinja service. |
| Action Result | The structured output from a scrape, including body content, status codes, and headers. |
| Input Schema | Parameters defining the target URL, proxy type, and browser headers. |
ScrapeNinja Advanced Features
- Automatic Proxy Rotation: Leverages ScrapeNinja's built-in residential and data center proxy pools.
- Transparent Credential Refresh: Membrane manages auth tokens automatically, ensuring long-running scraping scripts don't fail due to expired sessions.
- Direct Proxy Requests: Perform raw API calls through
membrane requestfor specialized endpoints not covered by standard actions. - Long-polling Status: Use the
--waitflag on connection tasks to handle asynchronous setup in automated CI/CD pipelines. - JSON-First Output: All commands support a
--jsonflag for seamless integration into downstream data pipelines.
SKILL.md
Loading
Related Openclaw Skills
Video WeChat IP Script Agent
A professional-grade content generation tool designed to create high-impact scripts, titles, and publishing packages specifically for WeChat Channels (Channels IP).
China Stock Analysis
A specialized AI agent skill designed to provide deep technical and fundamental analysis of Chinese A-shares, Hong Kong stocks, and US-listed Chinese companies.
Pre-Landing PR Review
A comprehensive AI-driven code review agent that analyzes PR diffs for structural integrity, data safety, and design consistency while proactively applying auto-fixes.
YouMind Illustration PPT Generator
A sophisticated AI agent skill that transforms text outlines into high-impact presentation decks featuring a consistent IP character and professional YouMind-style illustrations.
To-Do Accelerator
To-Do Accelerator is a powerful skill designed for seamless task hand-offs and collaborative project execution between human users and AI agents.
Deconsolidation Cargo Breakdown & Receiving
A specialized technical reference for managing container stripping, cargo breakdown, and cross-dock operations.
Featured*
