A privacy-centric web content extraction tool featuring local parsing and optional remote cleaning for optimized AI token usage.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install smart-web-fetch-safe
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install smart-web-fetch-safe using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Smart Web Fetch Safe?
Smart Web Fetch Safe is a specialized tool within the Openclaw Skills ecosystem designed for secure and efficient web content retrieval. It prioritizes user privacy by offering a default local parsing mode where data remains entirely on the local machine. By removing noise such as advertisements, navigation bars, and headers, it ensures that only relevant content is passed to AI agents, significantly reducing token consumption.
For users who require deeper content optimization, the skill also supports an optional remote cleaning mode via Jina Reader. This dual-mode approach allows developers to balance the need for absolute privacy against the benefits of high-efficiency 50-80% token reduction. With built-in domain white-listing and character limits, it provides a robust framework for controlled and safe automated web research.
Smart Web Fetch Safe Use Cases
- Extracting clean article text for AI summarization without sending sensitive URLs to third-party services.
- Reducing LLM costs by stripping boilerplate HTML and noise from web pages before processing.
- Building research agents that operate within a restricted list of approved domains for corporate compliance.
- Developing automated workflows that require fast, local HTML parsing without external API dependencies.
How Smart Web Fetch Safe Works
- The user provides a target URL and specifies the desired mode (local or remote) via the CLI or configuration.
- The skill checks the URL against the ALLOWED_DOMAINS whitelist to ensure security compliance.
- In local mode, the skill fetches the HTML using requests and parses it with BeautifulSoup4 to extract core content.
- In remote mode, the request is routed through a cleaning service to achieve maximum token optimization.
- Extracted content is truncated based on the MAX_CHARS limit to prevent context window overflow.
- The final cleaned content is returned as a plain string or structured JSON for the AI agent to consume.
Smart Web Fetch Safe Setup
To use this skill within the Openclaw Skills framework, ensure you have Python 3 installed and follow these steps:
# Install required dependencies
pip install beautifulsoup4 requests
# Set optional environment variables
export ALLOWED_DOMAINS="example.com,github.com,wikipedia.org"
export MAX_CHARS=5000
export DEFAULT_MODE=local
Smart Web Fetch Safe Data Schema & Taxonomy
The skill manages data through configuration variables and structured output formats to maintain consistency across Openclaw Skills integrations:
| Feature | Configuration Variable | Default Value |
|---|---|---|
| Domain Control | ALLOWED_DOMAINS | * (All allowed) |
| Output Length | MAX_CHARS | 10000 |
| Default Mode | DEFAULT_MODE | local |
Output can be retrieved as a raw string or a structured JSON object containing the cleaned text and metadata.
Smart Web Fetch Safe Advanced Features
- Dual-mode execution: Toggle between local privacy and remote deep-cleaning using simple CLI flags.
- Token Optimization: Achieves 50-80% reduction in content size when using remote cleaning mode.
- Security Hardening: Environment-level domain whitelisting to prevent unauthorized data access.
- Flexible Integration: Supports both standard text output and JSON format for easy programmatic consumption.
SKILL.md
Loading
Related Openclaw Skills
ElevenLabs Music Generation
A powerful AI agent skill that transforms text descriptions into fully produced songs, jingles, and soundtracks using the ElevenLabs Eleven Music API.
Product Description Generator
An advanced AI copywriting engine that creates platform-specific product listings using competitor research and FABE framework optimization.
Visual Note Card Generator
A high-performance skill for generating editorial-grade bilingual visual note cards and information-dense infographics with automated PNG export.
Video Converter
A high-performance media transcoding tool designed to transform incompatible video files into optimized, platform-ready delivery formats.
Reverse Costing
An analytical skill for deconstructing competitor products to expose their underlying cost structures and pricing strategies.
Crypto Arbitrage Monitor
A real-time monitoring tool that identifies price discrepancies for major cryptocurrencies across multiple exchanges to reveal arbitrage opportunities.
Featured*
