A comprehensive RSS collection and management system designed for incremental fetching, automated tagging, and professional HTML reporting.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install rss-fetcher
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install rss-fetcher using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is RSS Fetcher?
RSS Fetcher is a robust, unified system within the Openclaw Skills ecosystem designed to streamline the collection and organization of news feeds. It excels at incremental fetching, ensuring that only new content is processed by utilizing URL hashing for automatic deduplication. This approach minimizes redundant data and optimizes processing speed for high-volume information gathering.
Beyond simple retrieval, the skill features a sophisticated management layer that includes automated tagging, source health monitoring, and a category inheritance system. By generating filterable static HTML reports, developers and researchers can easily browse, search, and analyze large datasets of articles through a clean, multi-dimensional web interface.
RSS Fetcher Use Cases
- Building a centralized news hub for specific industries or technology sectors.
- Monitoring the availability and health of a large directory of RSS feeds.
- Creating automated, searchable archives of blog posts and articles for research.
- Generating clean, filterable daily briefing reports in static HTML format.
- Automating the categorization and tagging of news items for downstream AI processing.
How RSS Fetcher Works
- The system reads source configurations from a central JSON file defining names, URLs, and categories.
- The fetch engine initiates a multi-threaded process to pull XML data from the defined RSS feeds.
- Each article is checked against the database using a unique URL hash to ensure only new items are stored.
- An auto-tagging logic extracts categories from the RSS metadata or falls back to keyword extraction from titles.
- New data is committed to a local SQLite database with standardized Unix timestamps.
- A reporting script synthesizes the database content into a filterable HTML index for user interaction.
RSS Fetcher Setup
Follow these steps to initialize and run the RSS Fetcher within the Openclaw Skills framework:
# Navigate to the skill directory
cd skills/rss_fetcher
# Initialize the SQLite database structure
python3 scripts/init_db.py
# Add your sources to config/sources.json, then fetch articles
python3 scripts/fetch.py
# Generate the interactive HTML report
python3 scripts/generate_html.py
RSS Fetcher Data Schema & Taxonomy
The skill utilizes an efficient SQLite schema to manage high volumes of feed data. All timestamps are stored as INTEGER Unix values for performance.
| Table | Purpose | Primary Fields |
|---|---|---|
articles |
Main content storage | id, source_id, category, title, url, published_at |
tags |
Unique tag definitions | id, name |
article_tags |
Many-to-many relationship | article_id, tag_id |
fetch_logs |
Performance & health tracking | source_id, started_at, found, new, status |
RSS sources are managed externally in config/sources.json to allow for easy updates without database migrations.
RSS Fetcher Advanced Features
- Multi-threaded Execution: Support for up to 50 concurrent workers to speed up large-scale fetches.
- Health Check Utility: CLI tools to verify RSS source availability and view ingestion statistics.
- Custom Tagging Rules: Extensible
TAG_RULESlogic in Python for tailored keyword extraction. - Terminal Visualization: Built-in list tools to view recently fetched articles directly in the CLI as formatted tables.
- Advanced SQL Querying: Standardized database design allows for complex trend analysis and popular tag counting via standard SQL.
SKILL.md
Loading
Related Openclaw Skills
Weibo Hot Search Data Collector
A comprehensive multi-channel crawler and visualization engine for monitoring and archiving Weibo trending topics.
EverMemOS Memory System
A production-grade integration that provides AI agents with persistent long-term memory capabilities via the EverMemOS system.
Fullrun
An AI-driven CLI tool for diagnosing, optimizing, and managing Google Ads campaigns with automated triage and guardrail protection.
Kybera: AI Wallet Control & Token Research
An AI-powered assistant for deep token research, secure wallet management, and real-time DeFi operations across multiple chains.
Toutiao News Trends
A comprehensive data tool for fetching, archiving, and visualizing trending news and hot lists from Toutiao.
HttpSMS
A powerful SMS gateway integration that enables AI agents to send, receive, and manage text messages programmatically through a secure CLI interface.
Featured*
