A specialized tool that replaces standard web fetching with a high-efficiency Markdown cleaning pipeline for AI agents.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install smart-web-fetch
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install smart-web-fetch using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Smart Web Fetch?
Smart Web Fetch is a robust content retrieval tool designed to replace the default web_fetch functionality in AI workflows. It automatically processes target URLs through high-quality cleaning services like Jina Reader, markdown.new, and defuddle.md to provide agents with pure Markdown instead of cluttered HTML.
By utilizing this skill from the Openclaw Skills library, developers can significantly improve the quality of information provided to LLMs. The tool focuses on extracting the core text while stripping away navigation menus, advertisements, and scripts, leading to much higher accuracy during summarization and research tasks.
Smart Web Fetch Use Cases
- Reducing token consumption by stripping non-essential HTML from web articles.
- Implementing reliable web scraping for AI agents with multi-service redundancy.
- Automating research tasks where clean, formatted Markdown is required for documentation.
- Enhancing agent performance by providing context that is free of UI noise and tracking scripts.
How Smart Web Fetch Works
- The agent receives a URL and calls the Smart Web Fetch script.
- The system attempts to fetch the content via Jina Reader to generate high-quality Markdown.
- If the primary service fails, it automatically falls back to markdown.new or defuddle.md.
- If all cleaning services are unavailable, the skill retrieves the raw HTML as a final fallback.
- The cleaned content is returned to the agent, formatted specifically for LLM consumption.
Smart Web Fetch Setup
To integrate this skill into your environment, use the provided Python script. No API keys are required for the default cleaning services.
# Basic Markdown fetch
python3 scripts/fetch.py "https://example.com/article"
# Fetch with JSON metadata output
python3 scripts/fetch.py "https://example.com/article" --json
To ensure your agent prioritizes this Openclaw Skills implementation, update your openclaw.json to deny the built-in fetch tool:
{
"agents": {
"list": [
{
"id": "your-agent",
"tools": {
"deny": ["web_fetch"]
" }
}
]
}
}
Smart Web Fetch Data Schema & Taxonomy
The skill provides a structured response format to ensure compatibility with complex agentic workflows.
| Property | Description |
|---|---|
success |
Boolean flag indicating if the request was successful. |
url |
The specific proxy or cleaning service URL utilized. |
content |
The processed Markdown content of the target page. |
source |
The specific service that successfully provided the data (e.g., jina). |
error |
Contains error messages if all fallback attempts fail. |
Smart Web Fetch Advanced Features
- Four-level fallback strategy ensuring high availability for web content retrieval.
- Token optimization that saves between 50% and 80% on context costs per request.
- Native support for both plain text Markdown and structured JSON output formats.
- Zero-cost operation using free, high-performance web cleaning APIs.
SKILL.md
Loading
Related Openclaw Skills
Polymarket Prediction Market Integration
A powerful read-only integration for querying Polymarket prediction markets to retrieve real-time odds, volume, and event data.
Find Skills
A powerful discovery and management tool for finding and installing modular agent capabilities within the Openclaw Skills ecosystem.
Brek AI Partner Core Chat
A professional integration for the Brek Partner Core Chat API facilitating secure hotel discovery, multi-turn booking sessions, and payment orchestration.
SvelteKit Webapp Scaffold & Deploy
A comprehensive automation skill for scaffolding, configuring, and deploying opinionated SvelteKit PWAs with professional-grade defaults.
Alibaba Group Intelligence
A comprehensive intelligence skill providing deep insights into Alibaba Group's corporate history, business architecture, and strategic ecosystem.
ByteDance Comprehensive Insights
A comprehensive knowledge resource for analyzing ByteDance's business evolution, product ecosystem, and industry-leading recommendation algorithms.
Featured*
