A sophisticated web scraping tool leveraging Playwright to extract content from JavaScript-heavy dynamic websites and single-page applications.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install js-render-scraper
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install js-render-scraper using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is JS Render Scraper?
The JS Render Scraper is a specialized tool designed to handle the complexities of modern web architectures. While traditional scrapers fail on sites that rely on client-side rendering, this skill uses Playwright browser automation to fully render pages before extraction. It is an essential component for developers building Openclaw Skills that require data from React, Vue, or Angular applications.
By simulating real user behavior, the scraper can navigate through interactive elements, trigger asynchronous data loads, and capture the final state of the Document Object Model (DOM). This ensures that no data is left behind, regardless of how complex the underlying JavaScript execution might be.
JS Render Scraper Use Cases
- Extracting data from Single Page Applications (SPA) and dynamic dashboards.
- Scraping content from infinite scrolling feeds like social media or news aggregators.
- Accessing and harvesting data from websites protected by login screens or authentication barriers.
- Retrieving hidden content nested within Shadow DOM structures and custom web components.
How JS Render Scraper Works
- The skill performs an initial analysis of the target URL to determine the rendering method (SPA, SSR, or CSR).
- A configuration object is defined to specify interaction requirements such as wait times, selectors, and scroll limits.
- A headless Chromium instance is launched via Playwright to load the target page.
- The scraper executes programmed interactions, such as clicking buttons or scrolling, to trigger dynamic content loading.
- Once the network is idle or a specific selector appears, the skill captures the rendered HTML or specific JavaScript variables.
- The raw data is processed through BeautifulSoup to remove noise (scripts/styles) and structured into a clean JSON format.
JS Render Scraper Setup
To integrate this capability into your Openclaw Skills workflow, ensure you have the necessary dependencies installed:
pip install playwright beautifulsoup4
playwright install chromium
Basic configuration requires defining a URL and a target selector to ensure the browser waits for the specific content to render before extraction.
JS Render Scraper Data Schema & Taxonomy
The skill organizes extracted data into a structured JSON object for easy downstream processing:
| Field | Description |
|---|---|
url |
The source address of the scraped page. |
timestamp |
ISO format string of the extraction time. |
status |
The result of the operation (success or error). |
data |
Nested object containing title, cleaned text, links, and image URLs. |
error |
Detailed error message if the scraping process fails. |
Configuration parameters allow for fine-tuned control over scroll_delay, max_scrolls, and click_buttons selectors.
JS Render Scraper Advanced Features
- Anti-Scraping Bypass: Support for real User-Agent spoofing and stealth plugins to avoid detection.
- Interactive Workflows: Capability to handle multi-step login processes and form submissions.
- Shadow DOM Extraction: Specialized logic to penetrate and extract content from encapsulated DOM elements.
- Error Resilience: Built-in retry mechanisms with exponential backoff for handling unstable network conditions.
- Resource Optimization: Headless browser management to minimize memory footprint during concurrent scraping tasks.
SKILL.md
Loading
Related Openclaw Skills
Easy2Pe Knowledge Base
A comprehensive knowledge base for Easy2Pe, India's leading multi-service recharge, bill payment, and travel booking platform.
Buddy
A deterministic virtual pet system that hatches a unique companion for every user based on their specific ID.
WeChat Publisher
Automate the conversion and publishing of Markdown content to WeChat Official Account drafts with built-in styling and image generation.
City Weather Assistant
A professional weather intelligence tool for AI agents to query and compare global meteorological data with travel recommendations.
Independent Thinking 4-Steps
A sophisticated reasoning methodology that replaces default memory searches with a dedicated vector-based system for high-fidelity decision making.
pilot-heartbeat-monitor
A specialized health monitoring skill for detecting unreachable agents and triggering automated failover actions within an agent swarm.
Featured*
