An automated web scraping tool for AI agents that utilizes Playwright to extract structured content, text, and HTML from any URL.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install web-scraper-trae
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install web-scraper-trae using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Web Scraper Trae?
Web Scraper Trae is a robust browser automation tool designed to empower AI agents with the ability to crawl and extract information from the web. By utilizing the Playwright library, this Openclaw Skills integration can handle complex, modern websites including Single Page Applications (SPAs) that require JavaScript execution to display content. It provides a streamlined way to turn raw URLs into actionable data by capturing the page title, visible text, and full source code.
This skill is particularly valuable for workflows that require real-time data fetching, competitive research, or content aggregation. It operates in a headless browser environment, making it fast and suitable for various development setups while ensuring that the data retrieved is exactly what a human user would see in a standard browser window.
Web Scraper Trae Use Cases
- Extracting visible text from news articles or blog posts for AI summarization.
- Scraping technical documentation or wiki pages to provide context for coding tasks.
- Fetching the full HTML source code of a page for structural analysis or debugging.
- Automating data collection from dynamic websites that load content asynchronously via JavaScript.
How Web Scraper Trae Works
- The skill receives a URL as an input parameter from the user or agent.
- It initializes a headless Chromium browser instance using Playwright.
- The browser navigates to the specified URL and waits for the network to become idle to ensure all dynamic elements have loaded.
- It extracts specific metadata and content including the page title, all visible body text, and the raw HTML.
- The browser instance is closed to free up system resources.
- The extracted data is returned in a clean, structured JSON format for immediate use.
Web Scraper Trae Setup
To get started with this skill, you need to install Playwright and the Chromium browser engine in your environment:
npm install playwright
npx playwright install chromium
Once the dependencies are installed, you can implement the scraping script to work within your Openclaw Skills configuration, ensuring your agent has the necessary permissions to execute Node.js scripts and access the network.
Web Scraper Trae Data Schema & Taxonomy
Web Scraper Trae generates a structured output that makes it easy for AI agents to parse the information. The following data structure is returned after a successful scrape:
| Property | Description |
|---|---|
title |
The string value of the webpage document title. |
text |
The visible text content extracted from the body of the page. |
html |
The complete, rendered HTML source code of the page. |
url |
The original URL that was processed by the scraper. |
Web Scraper Trae Advanced Features
- Headless Mode: Runs efficiently in the background without a graphical user interface.
- SPA Compatibility: Handles dynamic content rendering by waiting for network idle states.
- Custom Timeouts: Includes a 60-second timeout to handle slow-loading or resource-heavy pages reliably.
- Environment Optimized: Uses specific browser flags like no-sandbox to ensure compatibility across different operating systems within Openclaw Skills.
- Data Extraction Precision: Automatically strips HTML tags for the text property to provide clean content for LLM consumption.
SKILL.md
Loading
Related Openclaw Skills
Free Subtitles — Generate and Burn In Captions
A powerful AI video editing skill that generates and burns subtitles into video files using cloud GPU acceleration.
AI Action Figure Generators CLI Skill
A specialized CLI tool that converts character illustrations or personal photos into realistic, collectible action figure renders.
SRT to Doc Converter
A comprehensive workflow that transforms raw SRT subtitle files into professionally structured Markdown documents with AI-powered analysis and research supplements.
GLM MCP Server Use
A comprehensive utility skill for installing, validating, and orchestrating official GLM MCP servers within the OpenClaw ecosystem.
Self-Improving Prompt
A dynamic skill that transforms vague or high-risk user prompts into clear, actionable, and verifiable instructions before execution.
Self-Improving Session
A specialized skill designed to capture durable workflow lessons and project conventions from active sessions to continuously refine AI behavior.
Featured*
