A high-performance CLI tool designed to scrape, crawl, and search the web, returning LLM-ready markdown content.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install web-researcher-mini
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install web-researcher-mini using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Firecrawl CLI?
Firecrawl CLI is a specialized tool built to bridge the gap between raw web content and AI model context windows. It excels at bypassing common web blocks and handling complex JavaScript rendering to deliver clean, structured markdown. By integrating this into your Openclaw Skills library, you can automate the extraction of documentation, research papers, and news directly into your development workflow.
The utility simplifies the data gathering process by stripping away non-essential elements like navigation bars and ads, ensuring that the AI agent receives only the most relevant information. Whether you are building a knowledge base or performing real-time web research, this skill provides the programmatic interface needed to scale web data ingestion safely and effectively.
Firecrawl CLI Use Cases
- Extracting technical documentation from library websites for coding context.
- Performing competitive intelligence by searching for specific market keywords.
- Mapping entire website architectures to identify specific content paths.
- Automating content monitoring for specific industries or research topics.
- Generating high-quality markdown datasets for RAG-based AI systems.
How Firecrawl CLI Works
- Initialize the environment by authenticating with the Firecrawl API and verifying credit limits.
- Dispatch search or scrape requests using the CLI commands to target specific URLs or search queries.
- The engine processes the target pages, rendering JavaScript and bypassing anti-bot measures in the cloud.
- Content is filtered to extract the main body, converting it into a clean markdown format or structured JSON.
- Output files are saved into a local directory for immediate consumption by the AI agent or other development tools.
Firecrawl CLI Setup
To get started with this skill, install the CLI and authenticate your environment:
npm install -g firecrawl-cli
firecrawl login --browser
firecrawl --status
Ensure a .firecrawl/ directory exists in your project to store scraped results, and add it to your .gitignore to prevent large data files from being committed.
Firecrawl CLI Data Schema & Taxonomy
The skill organizes its output within a localized directory structure to keep the project clean. Data is typically stored as follows:
| Data Type | Format | Common Extension | Description |
|---|---|---|---|
| Scraped Page | Markdown | .md | Cleaned text content optimized for LLMs. |
| Search Results | JSON | .json | Metadata, URLs, and snippets from web searches. |
| Site Map | Text/JSON | .txt / .json | A list of all discovered URLs on a specific domain. |
| Crawl Data | JSON | .json | Deep site data including links and structured content. |
Firecrawl CLI Advanced Features
- Parallel execution support using xargs for high-concurrency scraping tasks.
- Precise HTML filtering using include and exclude tags to target specific page elements.
- Geo-targeted search capabilities to retrieve localized results from specific countries.
- Time-based search filters (tbs) to find content from the last hour, day, or week.
- Deep crawling options with configurable depth limits and path-based inclusion rules.
SKILL.md
Loading
Related Openclaw Skills
Jira
A comprehensive integration for managing Jira Cloud issues, transitions, and time-tracking worklogs via a streamlined CLI interface.
Jira
A comprehensive integration for managing Jira Cloud issues, transitions, and worklogs through a CLI-based interface.
Memory Health Score
A comprehensive diagnostic utility that evaluates the integrity, freshness, and structure of an AI agent's memory system on a 100-point scale.
Memory Deduplication
A sophisticated utility for cleaning, merging, and consolidating redundant information in AI agent memory files to improve retrieval speed and accuracy.
Crypto & Precious Metals Price Monitor
A real-time price monitoring tool for cryptocurrency and precious metals with domestic and international market comparisons.
LinkedIn Human Warmup
Automate human-like LinkedIn interactions to safely warm up new or cold accounts using AdsPower and CDP.
METADATA
Featured*
