A robust tool for fetching and parsing web page content into structured Markdown or text via a secure, managed proxy.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install proxy-web-fetch
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install proxy-web-fetch using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Proxy Web Page Fetch?
The Proxy Web Page Fetch skill is a specialized utility designed to bridge the gap between AI agents and the live web. By routing requests through the OpenClaw Manager Web Fetch Proxy, it simplifies the process of retrieving web content by handling complex API key management and encrypted secrets internally. This ensures that developers can focus on processing content rather than managing authentication.
As a core part of the Openclaw Skills library, this tool provides a reliable way to transform any URL into machine-readable Markdown or plain text. Whether you need to scrape technical documentation, grab blog posts, or extract deep metadata, this tool handles the heavy lifting of web parsing with highly configurable options for caching, image retention, and link summaries.
Proxy Web Page Fetch Use Cases
- Reading and ingesting technical documentation directly from a live URL.
- Extracting clean, structured text from articles for AI-driven analysis or summarization.
- Scraping page metadata including titles, keywords, and descriptions for SEO research.
- Generating comprehensive summaries of web pages including image descriptions and link lists.
- Bypassing local environment restrictions by utilizing a centralized fetch proxy.
How Proxy Web Page Fetch Works
- The skill receives a target URL and optional formatting parameters from the user or agent.
- The request is routed to the central proxy server specified in the environment configuration.
- The OpenClaw Manager automatically injects required API keys from its encrypted secrets store.
- The proxy fetches the remote HTML content and parses it into the desired format (Markdown or text).
- A structured JSON response is returned, containing the parsed content, page title, and rich metadata for further processing.
Proxy Web Page Fetch Setup
To integrate this skill into your workflow, ensure the environment is correctly configured within the Openclaw Skills framework.
# Define the required proxy URL environment variable
export WEB_FETCH_PROXY_URL="https://your-manager-proxy-url.com"
# Ensure curl is available on your system
sudo apt-get install curl
# Test the skill using the provided wrapper script
./scripts/proxy_fetch.sh --url "https://example.com"
Proxy Web Page Fetch Data Schema & Taxonomy
The skill returns a structured JSON object containing the following primary fields:
| Field | Type | Description |
|---|---|---|
reader_result.content |
String | The main parsed content of the page in Markdown or Text. |
reader_result.title |
String | The extracted HTML title of the web page. |
reader_result.description |
String | A brief summary or meta description of the page. |
reader_result.metadata |
Object | Contains SEO data like keywords, viewport, and open-graph tags. |
reader_result.url |
String | The final canonical URL that was fetched. |
Proxy Web Page Fetch Advanced Features
- Internal Secret Management: Automatically handles API keys via the Manager's encrypted store without manual configuration.
- Custom Formatting: Supports both standard text and GitHub Flavored Markdown (GFM) output.
- Cache Control: Includes a
--no-cacheflag to force fresh fetches for dynamic content. - Content Filtering: Ability to strip images, retain data URLs, or generate dedicated link summaries.
- Extended Timeouts: Configurable request timeouts for fetching heavy or slow-loading enterprise documentation.
SKILL.md
Loading
Related Openclaw Skills
A-Share Stock Data Collector
A high-performance data acquisition tool designed to build and maintain a local SQLite database of Chinese A-share market historical K-line data.
Spaced Repetition Teaching (FSRS-6)
An adaptive spaced repetition engine utilizing the state-of-the-art FSRS-6 algorithm to optimize flashcard reviews for long-term retention.
Digital Labour
A comprehensive suite of 24 specialized AI agents designed to automate business back-office operations through a unified API.
ArbiScan - Cross-Exchange Crypto Scanner & Monitor
ArbiScan is a comprehensive read-only market scanner that identifies arbitrage opportunities and trading signals across four major cryptocurrency exchanges.
fundnews-search
An automated news aggregator that scrapes major Chinese financial platforms for mutual fund industry updates and generates structured Word reports.
Chinese Voice Skill
A professional-grade text-to-speech skill that leverages Microsoft Edge TTS to generate natural Chinese voice responses for AI agents.
Featured*
