A high-performance Docker-based web scraper utilizing Crawlee and Playwright to extract structured data from Amazon, YouTube, and dynamic web pages.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install deep-scraper-amazon
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install deep-scraper-amazon using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Deep Scraper?
Deep Scraper is a robust, containerized data extraction tool designed for high-performance web harvesting, serving as a critical component in the Openclaw Skills ecosystem. By combining Docker isolation with the power of Crawlee and Playwright, it provides a stable environment for bypassing common anti-scraping measures and rendering complex, JavaScript-heavy websites.
This skill is engineered for precision, offering specialized handlers that understand the unique structures of major platforms like Amazon and YouTube. Whether you are a developer looking to feed an AI agent with fresh market data or a researcher gathering content from across the web, Deep Scraper provides the reliability and structure needed for professional-grade data collection within the Openclaw Skills framework.
Deep Scraper Use Cases
- Tracking Amazon Best Sellers, New Releases, and Movers and Shakers across various categories.
- Conducting competitor research by extracting ASIN, BSR, and pricing data from product detail pages.
- Monitoring monthly sales velocity using the boughtPastMonth metric from Amazon search results.
- Fetching transcripts and descriptions from YouTube videos for content analysis or summarization.
- Crawling generic dynamic websites to extract clean, noise-free text data for LLM processing.
How Deep Scraper Works
- The tool identifies the target URL type to automatically select between Amazon, YouTube, or Generic scraping modes.
- It initializes a Docker container to ensure a clean, isolated environment for every request, preventing fingerprint tracking.
- For Amazon, it parses specific DOM elements to extract e-commerce metadata like rankings and sales volume.
- For YouTube, it intercepts network requests to capture high-accuracy timed transcripts directly from the API.
- For generic sites, Playwright renders the full page, scrolls to trigger lazy-loaded content, and extracts the core text.
- The extracted data is cleaned of noise and returned as a structured JSON object for immediate use in other Openclaw Skills workflows.
Deep Scraper Setup
Before using Deep Scraper, ensure that Docker is installed and running on your host machine. You must first build the container image locally:
docker build -t clawd-crawlee skills/deep-scraper/
Once built, you can trigger the scraper via the command line for various targets:
# Scrape Amazon Best Sellers
docker run -t --rm clawd-crawlee node assets/amazon_handler.js "https://www.amazon.com/zgbs/electronics"
# Scrape YouTube Transcripts
docker run -t --rm clawd-crawlee node assets/main_handler.js "https://youtube.com/watch?v=VIDEO_ID"
Deep Scraper Data Schema & Taxonomy
The skill produces a standardized JSON output. The structure adapts based on the scraping mode used:
| Property | Description |
|---|---|
| status | Indicates if the scraping task was a SUCCESS or ERROR. |
| type | Identifies the handler used (e.g., bestsellers, search, product-detail, TRANSCRIPT, GENERIC). |
| products | An array of objects for Amazon mode, including title, asin, price, rating, and reviews. |
| boughtPastMonth | Specific to Amazon search/detail pages, showing recent sales volume. |
| data | The primary text content or transcript payload for Generic and YouTube modes. |
| scrapedAt | ISO timestamp of when the data was captured. |
This structured format ensures seamless integration with other tools in the Openclaw Skills catalog.
Deep Scraper Advanced Features
- Intelligent handler routing that automatically maps URLs to specialized scraping logic.
- Dynamic content handling using Playwright to wait for JS execution and lazy loading.
- Multi-page support for Amazon search results to collect larger datasets in a single command.
- Anti-detection features including automatic cookie clearing and browser fingerprint randomization.
- Smart noise reduction in generic mode that extracts core document text while filtering out navigation and ads.
SKILL.md
Loading
Related Openclaw Skills
ClawPrompt
A browser-based smart teleprompter featuring mobile remote control and real-time dual-screen synchronization for professional video recording.
Deep Research
A sophisticated research framework for AI agents to perform multi-source web investigation, fact-checking, and structured synthesis.
Huawei Cloud Architecture & Terraform Pro
A comprehensive automation skill for designing Huawei Cloud infrastructures and generating production-ready Terraform templates.
Plashboard Admin
An autonomous administrative tool for managing Plashboard templates and converting natural language requests into live dashboards.
Horse Sticker Maker
A specialized web application generator for creating custom, WeChat-compatible animated gold horse GIF stickers for Chinese New Year 2026.
Vercel to Cloudflare Worker Migration
A technical automation skill for migrating Next.js applications from Vercel to Cloudflare Workers with native Supabase and Hyperdrive support.
Featured*
