An automated news fetching and summarization tool designed to extract content from RSS feeds and websites to generate structured reports.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install news-crawler
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install news-crawler using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is News Crawler?
The News Crawler is a sophisticated automation skill developed for Openclaw Skills that streamlines the process of staying informed. It allows users and AI agents to monitor specific RSS feeds or targeted URLs, extract full-text content, and leverage Large Language Models to generate concise article summaries.
This skill bridges the gap between raw data sources and actionable intelligence by transforming cluttered web pages into clean, structured JSON data. It is particularly effective for those building automated newsletters, research assistants, or competitive intelligence monitors within the Openclaw Skills ecosystem.
News Crawler Use Cases
- Generating automated daily news briefings from multiple global technology or finance sources.
- Monitoring specific industry developments by tracking custom RSS feeds in real-time.
- Batch processing specific article URLs to extract core content for research databases.
- Creating condensed summaries of lengthy news pieces to save reading time.
How News Crawler Works
- The user identifies a news source via a standard RSS URL or a direct web link.
- The RSS fetcher script identifies the most recent articles and retrieves their metadata.
- The crawler script navigates to the article URLs to extract the full body content while bypassing UI clutter.
- Extracted content is processed by an LLM to distill the core message into a summary.
- The final output is organized into a structured report sorted by category or time.
News Crawler Setup
To get started with this skill for Openclaw Skills, use the following Python scripts:
To fetch a list of articles from an RSS feed:
python3 scripts/rss_fetcher.py https://www.solidot.org/index.rss 5
To crawl and extract content from a specific news URL:
python3 scripts/crawl.py https://example.com/news/article.html 3000
News Crawler Data Schema & Taxonomy
The skill produces structured JSON data for both discovery and extraction phases:
| Component | Data Fields |
|---|---|
| RSS Fetcher | title, link, description, published, count |
| Content Crawler | url, title, content, length |
Content is primarily handled in UTF-8 encoding with a default truncation limit of 5000 characters to ensure compatibility with most LLM context windows.
News Crawler Advanced Features
- Support for major global and regional feeds including Hacker News, TechCrunch, and 36Kr.
- Configurable content length parameters to manage API token usage during summarization.
- Extensible design allowing for integration with Cron for scheduled tasks.
- Hook-ready architecture for pushing summaries to external platforms like Feishu, Slack, or Email.
SKILL.md
Loading
Related Openclaw Skills
sense-music
A music perception skill for AI agents to analyze BPM, key, structure, genre, mood, and lyrics from audio files.
Ecommerce Copy Humanizer ZH
A sophisticated tool designed to strip away the robotic tone of AI-generated Chinese e-commerce copy, replacing it with natural, high-conversion human-like expression.
Self Skill Evolver
A sophisticated Python CLI tool designed to monitor, analyze, and optimize the lifecycle and performance of OpenClaw agent skills.
Content Source to Markdown
A specialized tool designed to transform noisy web content, social media posts, and URLs into structured, reusable markdown briefs for ecommerce intelligence.
Nika Skill Creator
A specialized framework for engineering, refactoring, and validating AI agent skills compliant with the Nika platform constraints.
Critic Agent Skill
A specialized agent for reviewing, scoring, and providing actionable feedback on AI-generated outputs using a structured rubric.
Featured*
