A deterministic tool for converting complex webpages into clean, LLM-ready markdown with intelligent fallback mechanisms.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install web-markdown-navigator
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install web-markdown-navigator using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Web Markdown Navigator?
Web Markdown Navigator is a specialized utility within the Openclaw Skills ecosystem designed to solve the problem of noisy web data. Instead of passing raw, unoptimized HTML to an AI agent, this skill uses a sophisticated multi-layer extraction process. It leverages Mozilla Readability and Turndown to distill a webpage down to its core content, ensuring that the resulting markdown is focused, token-efficient, and easy for language models to process.
This skill is particularly effective for developers building research agents or automated content pipelines. By prioritizing clean text over raw structure, it enhances the accuracy of summarization and data extraction tasks. As a part of the Openclaw Skills collection, it provides a reliable interface for turning any public URL into structured documentation.
Web Markdown Navigator Use Cases
- Extracting clean article content for automated summarization workflows.
- Converting documentation pages into markdown for RAG (Retrieval-Augmented Generation) datasets.
- Automating research tasks where raw HTML noise interferes with AI reasoning.
- Scrapping blog posts or news sites for data analysis while excluding navigation and ads.
How Web Markdown Navigator Works
- The skill accepts a URL input and optional parameters like max-chars and timeout-ms.
- It performs immediate URL safety checks to prevent access to localhost or private IPv4 addresses.
- In Layer 1, the script attempts to extract content using Mozilla Readability and converts it via Turndown.
- If the initial extraction results in thin content or boilerplate (often the case with SPAs), the skill triggers a Layer 2 fallback.
- The system can utilize a browser tool fallback to capture rendered content if the static fetch fails to provide quality markdown.
- The final output is returned as clean markdown, including the source URL and the specific extraction method used.
Web Markdown Navigator Setup
To integrate this skill into your local environment, navigate to the scripts directory and execute the runner using Node.js:
cd skills/web-markdown-navigator/scripts
node fetch-markdown.mjs "<url>" --max-chars 50000
Ensure your environment has the necessary dependencies installed as specified in the Openclaw Skills repository guidelines.
Web Markdown Navigator Data Schema & Taxonomy
The Web Markdown Navigator organizes its output to be highly predictable for downstream AI agents. The data is typically returned with the following taxonomy:
| Property | Description |
|---|---|
| Content | The primary body text converted to markdown format. |
| Source URL | The original web address used for the extraction. |
| Method | A label indicating if readability or fallback-text was utilized. |
| Truncation Flag | A note appended to the content if it was cut short by the max-chars limit. |
| Error Codes | Standardized exit codes (1-4) for troubleshooting arguments, blocks, or network failures. |
Web Markdown Navigator Advanced Features
- Smart Layered Extraction: Automatically switches between Readability and text-only fallbacks based on content density.
- Configurable Constraints: Set strict character limits and timeouts to manage token usage and performance within Openclaw Skills.
- Security Hardening: Integrated URL safety filters to prevent SSRF (Server-Side Request Forgery) by blocking private network ranges.
- JSON Output Mode: Support for the --json flag to facilitate programmatic consumption by other automated agents.
SKILL.md
Loading
Related Openclaw Skills
Report Generator
An automated reporting engine designed to transform structured data files into professional, decision-ready executive summaries and visualizations.
Recursive Swarm
A bounded recursive orchestration skill that decomposes massive tasks into manageable, trackable sub-task trees with explicit state management.
OpenRouter Analytics
A comprehensive monitoring tool for tracking OpenRouter API consumption, costs, and request-level diagnostics.
OpenClaw Healthcheck Cron
A portable automation framework for scheduling deep system health audits and sanitizing environment data for secure reporting.
Workflow Orchestrator
A structural skeleton for managing deterministic, queue-driven AI orchestration and multi-phase state transitions.
TBOT Controller
A specialized automation interface for the TradingBoat stack that prioritizes DB-first queries and secure lifecycle management.
Featured*
