A versatile tool for archiving URLs and screenshots into a tag-first, searchable personal knowledge base.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install knowledge-base-collector
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install knowledge-base-collector using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Knowledge Base Collector?
The Knowledge Base Collector is a powerful utility within the Openclaw Skills ecosystem designed to help developers and researchers preserve digital content. It captures data from diverse sources including standard web pages, X (Twitter) threads, and WeChat Official Accounts, transforming them into organized Markdown files and structured JSON metadata. By centralizing information into a shared local directory, it ensures that your digital assets remain accessible, searchable, and ready for AI-driven retrieval even if the original sources go offline.
This skill focuses on a tag-first classification system, making it ideal for building a comprehensive research library. It bridges the gap between raw web browsing and structured knowledge management by providing automated ingestion, metadata extraction, and integrated search capabilities tailored for modern AI workflows.
Knowledge Base Collector Use Cases
- Saving technical documentation and blog posts for offline reference and deep reading.
- Archiving X/Twitter threads or WeChat articles that are prone to deletion or access restrictions.
- Converting visual information from screenshots into searchable text records using OCR.
- Building a long-term research database for Openclaw Skills to reference during complex task execution.
- Generating weekly summaries and topical reports from collected insights.
How Knowledge Base Collector Works
- The user sends a URL, screenshot, or content snippet via a command-line interface or a connected messaging platform like Telegram.
- The skill analyzes the input type and selects the appropriate ingestion script (URL or Image).
- For URLs, it utilizes tools like Jina Reader to extract the primary text; for WeChat links, it can offload the task to a connected macOS node to bypass cloud-fetch blocks.
- For images, the system extracts text using Tesseract OCR or multimodal LLMs.
- The processed content is saved to a date-structured folder containing a content.md file and a meta.json metadata file.
- The global index.jsonl file is updated to ensure the new entry is immediately available for search and retrieval.
Knowledge Base Collector Setup
To get started with this knowledge collection tool, ensure your environment is configured. You can manually ingest content using the following Openclaw Skills commands:
To archive a URL:
python3 /home/ubuntu/.openclaw/skills/knowledge-base-collector/scripts/ingest_url.py "<URL>" --tags "#coding #ai" --note "Research notes"
To archive an image with OCR text:
python3 /home/ubuntu/.openclaw/skills/knowledge-base-collector/scripts/ingest_image.py /path/to/image.jpg --text-file /path/to/ocr.txt --title "System Architecture" --tags "#devops"
Knowledge Base Collector Data Schema & Taxonomy
The skill organizes data within the default root at /home/ubuntu/.openclaw/kb. The hierarchy is designed for both human readability and machine parsing:
| Component | Path / Format | Description |
|---|---|---|
| Global Index | urls/index.jsonl |
A line-delimited JSON file for fast searching. |
| Monthly Folder | urls/<YYYY-MM>/ |
Time-based partitioning of knowledge items. |
| Item Folder | <item-id>/ |
Contains content.md, meta.json, and optional images. |
| Metadata | meta.json |
Stores source URL, timestamp, tags, and status. |
| Content | content.md |
The cleaned, Markdown-formatted body of the source. |
Knowledge Base Collector Advanced Features
- WeChat Anti-Blocking: Leverages connected macOS nodes (via nodes.run) to fetch WeChat content when standard cloud requests are flagged.
- Integrated Search: Use
search_kb.pyto query your knowledge base by tags, keywords, or source directly through Openclaw Skills. - Multi-Agent Compatibility: Designed to support Telegram Q&A flows, allowing other agents to query your personal library for context.
- Weekly Digests: Includes a specialized script to generate weekly summaries of new entries for knowledge synthesis.
- Automatic Redaction: Provides a framework for stripping sensitive tokens and secrets before content is saved to the disk.
SKILL.md
Loading
Related Openclaw Skills
DB Read-Only Skill
A secure interface for executing read-only SQL queries against PostgreSQL and MySQL databases without risk of data modification.
Claw Roam
A Git-powered synchronization engine to keep your AI agent workspace consistent across local machines and remote servers.
Bitfinex Lending Monitor
Automate the monitoring and reporting of Bitfinex funding interest and wallet balances through secure API integration.
Agent Dashboard Sync
A high-performance, no-LLM synchronization pipeline to monitor AI agent heartbeats and runtime status via Cloudflare KV and Worker APIs.
Reminder Secretary
A natural-language personal assistant that captures events into a git-synced workspace and schedules automated Telegram notifications.
Seedance 3x3 Optimizer
An AI-powered script optimizer that converts narrative plots into structured 9-segment visual scripts specifically for Seedance video generation.
Featured*
