An industrial-grade search aggregator that combines multiple data providers into a single, sanitized, and structured JSON output.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install search-cluster
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install search-cluster using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Search Cluster?
Search Cluster is a high-availability search aggregation tool specifically designed for developers building sophisticated workflows with Openclaw Skills. By unifying results from official APIs like Google Custom Search and Wikipedia alongside community data from Reddit and stealth scraping via DuckDuckGo, it provides a comprehensive information retrieval layer. The tool is built with an industrial standard v3.1 architecture, prioritizing subprocess isolation and strict security protocols to ensure data integrity and agent safety.
Search Cluster Use Cases
- Conducting deep-dive market research across multiple search engines and news aggregators.
- Automating information gathering for Openclaw Skills from community forums like Reddit and factual databases like Wikipedia.
- Implementing secure, stealth-based web scraping for competitive analysis without triggering bot detection.
- Real-time monitoring of global trends through Google News RSS integration.
How Search Cluster Works
- The user or AI agent initiates a query through the scripts/search-cluster.py entry point.
- The system identifies active providers based on the environment configuration, such as Google CSE or the Scrapling venv.
- Search queries are dispatched; stealth requests are routed through a dedicated fetcher to ensure subprocess isolation.
- All retrieved data passes through a native internal scrubber to perform path neutralization and snippet sanitization.
- The aggregated results are formatted into a clean JSON structure, optionally cached via Redis for high-speed subsequent access within Openclaw Skills.
Search Cluster Setup
To deploy Search Cluster for your Openclaw Skills, set up the Scrapling provider environment:
# Create a dedicated virtual environment
python3 -m venv venv/scrapling
# Install the Scrapling dependency
venv/scrapling/bin/pip install scrapling
# Configure the environment variable path
export SCRAPLING_PYTHON_PATH="/path/to/venv/scrapling/bin/python"
Additionally, configure GOOGLE_API_KEY and GOOGLE_CSE_ID in your vault for official Google search results.
Search Cluster Data Schema & Taxonomy
Search Cluster organizes aggregated data into a uniform schema to ensure compatibility across different Openclaw Skills. The output follows this structure:
| Attribute | Description |
|---|---|
| source | The origin provider (e.g., google, wiki, reddit, gnews) |
| title | The headline or page title of the result |
| link | The validated URL to the source content |
| snippet | A sanitized, path-neutralized text excerpt for LLM consumption |
All data is processed with strict TLS verification and internal scrubbing.
Search Cluster Advanced Features
- Subprocess isolation for stealth_fetch.py to maximize security during headless scraping.
- Integrated Redis support for high-performance result caching, reducing latency for repetitive Openclaw Skills queries.
- Custom User-Agent string configuration to emulate specific browser environments.
- Multi-provider 'all' execution mode for comprehensive data coverage in a single command.
SKILL.md
Loading
Related Openclaw Skills
Pi-hole Controller (pihole-ctl)
A powerful management and monitoring skill for local Pi-hole instances that provides deep insights and control via CLI and FTL database queries.
Node-RED Manager
A powerful management tool for Node-RED instances that enables automated flow deployment, node management, and runtime diagnostics via a streamlined CLI.
Newman - Postman CLI Runner
An automated command-line tool for running Postman collections to enable seamless API testing and reporting.
Mermaid Architect
A specialized agent skill for generating robust, visually appealing Mermaid.js diagrams using safe syntax and professional layouts.
Security Guardian
Automated security auditing system for detecting hardcoded secrets and container vulnerabilities in development projects.
Stability AI Image Generation
Generate high-quality AI images using SDXL and Stable Diffusion 3 directly through your command line or agent workflow.
METADATA
Featured*
