An AI-powered autonomous agent that navigates complex, multi-page websites to extract highly structured JSON data based on custom schemas.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install firecrawl-agent
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install firecrawl-agent using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Firecrawl Agent?
The firecrawl-agent is an advanced, AI-driven web scraper designed to navigate deep site hierarchies, discover relevant data paths, and return beautifully structured JSON payloads. Unlike standard scraping utilities that target single pages with static selectors, this tool empowers an autonomous agent to execute multi-page research to satisfy complex data demands.
By leveraging cutting-edge LLMs, this skill figures out site architecture dynamically. Whether you need pricing structures, multi-level catalogs, or directory listings, the agent handles the heavy lifting, saving hours of manual scraping setup. It represents a significant leap forward in browser automation capabilities for developers building modern data pipelines using Openclaw Skills.
Firecrawl Agent Use Cases
- Extracting Complex Multi-Page Directories: Autonomously traversing nested links to gather directory records.
- Pricing Competitor Analysis: Pulling tiered pricing data, feature comparisons, and localized pricing matrices from SaaS or eCommerce platforms.
- Product Catalog Synthesis: Aggregating high-fidelity product listings, descriptions, and specifications into clean JSON.
- Schema-Based Structured Extraction: Converting unstructured web pages into predefined JSON schemas for effortless database ingestion.
How Firecrawl Agent Works
- Initiation: The agent receives a natural language prompt, a starting URL, and optionally a structural JSON schema.
- Analysis & Exploration: The agent crawls the initial target pages, identifying navigation patterns, pagination buttons, and relevant links.
- Extraction & Refinement: The underlying AI model parses the page DOM, filtering raw HTML into the desired attributes mapped to your schema.
- Structured Compilation: The output is accumulated, structured, and exported directly into a clean output file as defined by the CLI options.
Firecrawl Agent Setup
Install the Firecrawl CLI and run the agent using the following terminal commands:
# Standard structured extraction with wait flag to block execution until finished
firecrawl agent "extract all pricing tiers" --wait -o .firecrawl/pricing.json
# Advanced extraction with a defined JSON schema
firecrawl agent "extract products" --schema '{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"number"}}}' --wait -o .firecrawl/products.json
# Targeting specific domain structures
firecrawl agent "get feature list" --urls "<url>" --wait -o .firecrawl/features.json
Make sure to have your target execution environment configured. This command can also be run via npx using npx firecrawl agent.
Firecrawl Agent Data Schema & Taxonomy
The firecrawl-agent outputs data dynamically depending on your specified options and schema configurations. Below are the key structural details:
Available Options
| Option | Description | Target Use Case |
|---|---|---|
--urls <urls> |
Comma-separated starting URLs | Targets specific entrypoints |
--model <model> |
LLM selection: spark-1-mini or spark-1-pro | Fine-tunes speed vs depth |
--schema <json> |
Raw JSON schema input | Forces predictable, strict output structures |
--schema-file <path> |
Path to a local JSON schema file | Manages complex data schemas cleanly |
--max-credits <n> |
Restricts credit usage per run | Keeps API and scraping costs optimized |
--wait |
Sync blocks and waits for completion | Useful for sequential CI/CD pipelines |
-o, --output <path> |
File destination for results | Direct exports to projects |
Firecrawl Agent Advanced Features
- Dynamic Multi-Page Navigation: Rather than halting at a single page, the agent proactively navigates nested menus and pagination links.
- Strict Schema Enforcement: Pair your requests with standard JSON schemas to ensure immediate database compatibility without manual cleaning.
- Cost Control Guardrails: Apply the
--max-creditsparameter to programmatically restrict the deep-crawl capabilities from exhausting resources. - Flexible Model Selection: Toggle between lightweight speed-optimized models and high-reasoning models (spark-1-pro) depending on target site complexity.
SKILL.md
Loading
Related Openclaw Skills
Firecrawl Build Onboarding
A professional automation skill designed to seamlessly onboard Firecrawl credentials, initialize environment configurations, and install required SDKs in any codebase.
Firecrawl Build Interact
Integrate the Firecrawl /interact API endpoint into your application code to handle dynamic pages, pagination, forms, and authenticated sessions.
Firecrawl Build Scrape
An API integration skill designed to cleanly extract markdown, HTML, screenshots, and metadata from single, known URLs.
Agent OS
Agent OS is a lightweight, file-based collaboration layer that enables seamless, auditable, and structured multi-agent coordination without relying on bloated chat histories.
Global Energy & Natural Resources Intelligence
A massive, data-driven intelligence skill covering the full energy stack from hydrocarbons to renewables and critical minerals.
Global Food & Agriculture Intelligence
A comprehensive agrifood intelligence skill connecting 22 data sources spanning food security, crop production, commodity prices, climate impacts, and trade flows.
Featured*
