A high-performance web content extraction engine that transforms cluttered HTML into clean, structured Markdown for AI agents and developers.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install defuddle-extractor
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install defuddle-extractor using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Defuddle Web Content Extractor?
Defuddle is a specialized tool designed to bridge the gap between complex web structures and the clean text formats required for modern AI processing. By utilizing the Defuddle library, this skill identifies the primary content of any webpage while automatically stripping away distracting elements like advertisements, sidebars, and comments. This ensures that only relevant information is passed to your models or storage systems.
Built to be versatile, it supports both command-line interactions and deep Node.js integration. Within the Openclaw Skills ecosystem, Defuddle serves as a critical component for developers building RAG pipelines, automated research tools, or content aggregators that require high-quality, distraction-free Markdown output.
Defuddle Web Content Extractor Use Cases
- Extracting clean article text for AI-powered summarization and analysis.
- Converting documentation or blog posts into Markdown for personal knowledge management.
- Automating the delivery of web content to communication platforms like WeChat or Telegram.
- Scraping high-quality text data for training or fine-tuning LLM-based applications.
How Defuddle Web Content Extractor Works
- The tool accepts a target URL or a local HTML file as input through the CLI or API.
- The underlying engine parses the DOM structure and applies heuristics to detect the main content body.
- Non-essential elements such as scripts, ads, and navigation menus are pruned to clean the data.
- The processed content is converted from HTML to Markdown format for maximum readability.
- The final output is delivered as a Markdown string, a JSON object with metadata, or piped to another script.
Defuddle Web Content Extractor Setup
To get started with this skill, ensure you have Node.js 18 or higher installed on your system. You can then install the package globally using npm:
npm install -g defuddle
Once installed, you can verify the installation by running:
defuddle --version
Defuddle Web Content Extractor Data Schema & Taxonomy
The skill produces structured data that can be output in various formats. Below is the schema for the JSON output mode:
| Attribute | Description |
|---|---|
| title | The extracted title of the webpage or article. |
| content | The cleaned HTML version of the primary content. |
| contentMarkdown | The primary content converted into optimized Markdown. |
| metadata | Includes extracted metadata like author, date, and original URL. |
| selectors | Custom CSS selectors used during the extraction process (if provided). |
Defuddle Web Content Extractor Advanced Features
- Custom Content Selectors: Manually define which parts of a page to extract for complex layouts.
- Debug Mode: Enable verbose logging to troubleshoot extraction issues on non-standard websites.
- Multi-Platform Integration: Pre-built shell scripts for sending extracted content directly to WeChat or Telegram.
- Configurable Filtering: Options to specifically remove images, hidden elements, or specific CSS classes.
- Programmatic API: Full support for Node.js integration using JSDOM for advanced automation within Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
Self-Audit
A diagnostic observability skill that tracks and analyzes AI agent tool calls to distinguish between necessary actions and reflexive API invocations.
Group Director
A specialized video execution layer that converts chat-derived prompts into high-quality short videos for enterprise communication platforms.
AI Intelligent Knowledge QA
A sophisticated AI-driven Q&A system combining semantic search and GPT integration to automate FAQ management and knowledge retrieval.
EdrawMind Mind Map Generation
A powerful integration that transforms structured Markdown or natural language into professional, editable EdrawMind maps with custom layouts and styles.
AkShare Fund Analysis
A quantitative data analysis tool for public funds, enabling deep insights into holdings, valuations, and performance tracking via AkShare.
Free Web Search Ultimate
A universal Search-First knowledge acquisition plugin that enables LLMs to access real-time web data, news, and full-page content.
Featured*
