An integration for extracting structured data from raw web content, text, or markdown using the Scrapfly Extraction API and AI-driven models.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapfly-extraction
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapfly-extraction using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapfly Extraction?
The scrapfly-extraction tool is a utility designed to streamline how developers harvest structured information from raw web pages, documents, and markdown content. By leveraging the Scrapfly Extraction API alongside the Python SDK, this skill processes unstructured data into well-defined schemas. It minimizes the need for manual parsing scripts, providing a programmatic interface for retrieving clean data arrays, key-value pairs, or document answers within Openclaw Skills projects.
This utility operates through three main mechanisms: natural language prompts directed at LLMs, pre-trained Auto AI models for specific content types, and rigid custom CSS/XPath templates. When integrated within a broader agentic workflow, this skill enables AI agents to read, query, and extract metadata from diverse web sources with high precision.
Scrapfly Extraction Use Cases
- Extracting structured product, review, article, or real estate data from raw HTML without writing custom regex.
- Querying web documents using natural language to get direct answers and insights from target web pages.
- Standardizing scraped data into rigid JSON formats to ingest directly into analytics pipelines or databases.
- Processing markdown or plain text files in bulk using targeted extraction prompts within Openclaw Skills integrations.
- Integrating automated extraction rules for programmatic web scraping pipelines.
How Scrapfly Extraction Works
- Input Preparation: The user or AI agent captures raw content (HTML, XML, markdown, or plain text) manually or via an initial scraping action.
- Configuration Definition: The agent instantiates an ExtractionConfig object specifying the raw body, its content type, and exactly one extraction method (prompt, pre-trained model, or template).
- API Dispatch: The Scrapfly SDK transmits the structured request payload to the Scrapfly Extraction endpoint using the configured API key.
- Structured Parsing: Scrapfly processes the input content according to the requested method, leveraging either LLMs or strict CSS/XPath selector trees.
- Response Delivery: The structured payload is returned as a clean dictionary, containing parsed data fields and extraction metadata.
Scrapfly Extraction Setup
To use this skill within Openclaw Skills, ensure the official Python SDK is installed and configured:
pip install scrapfly-sdk
Next, authenticate by exporting your Scrapfly API key as an environment variable:
export SCRAPFLY_API_KEY="your_scrapfly_api_key_here"
Alternatively, you can provide the API key directly to the ScrapflyClient instance when initializing it in your scripts.
Scrapfly Extraction Data Schema & Taxonomy
Config Parameters
| Parameter | Type | Description |
|---|---|---|
body |
str | Raw content to extract from (HTML, markdown, plain text, XML). |
content_type |
str | Format of the input document (e.g., 'text/html', 'text/markdown'). |
extraction_prompt |
str | Natural language prompt directing the AI extraction. |
extraction_model |
str | Pre-trained model identifier (e.g., 'product', 'article', 'review_list'). |
extraction_template |
str/dict | A saved dashboard template slug or an inline CSS/XPath template dictionary. |
Response Output Schema
Responses returned by the Scrapfly client contain structured fields:
extraction_result: A dictionary containing metadata and the extracted payload.data: The raw parsed data extracted from the document, formatted as JSON or structured lists.content_type: The MIME type of the extracted output payload.
Scrapfly Extraction Advanced Features
- Inline Scrape Extraction: Run extraction tasks directly within scraping steps by attaching extraction prompts or models to ScrapeConfig, minimizing latency and API roundtrips.
- Saved and Versioned Templates: Refer to templates stored on the Scrapfly platform via slug names, featuring full version control and URL domain/path matching.
- Pre-trained Auto AI Models: Instantly access robust standard models for common entities such as product listings, job postings, organizations, and social media posts.
- Error Resiliency: Robust error-handling structures using the native ScrapflyError exception wrapper within Openclaw Skills agent environments.
SKILL.md
Loading
Related Openclaw Skills
Scrapfly TypeScript SDK
A TypeScript and JavaScript SDK for Scrapfly designed to handle complex web scraping, JavaScript rendering, anti-bot bypass, and AI-driven data extraction.
Scrapfly Alerting
A threshold-based monitoring integration to watch Scrapfly account metrics and trigger notifications via webhooks, email, or in-app alerts.
Scrapfly Go SDK
An advanced web scraping SDK for Go that enables seamless proxy rotation, anti-bot bypass, JavaScript rendering, and automated data extraction.
Scrapfly Cloud Browser
Automate cloud browsers using the Scrapfly Cloud Browser API with Python Playwright for residential proxy rotation and anti-bot fingerprint management.
Scrapfly Scraper
An advanced web scraper leveraging the Scrapfly API to extract web data with anti-bot bypass, proxy rotation, and headless browser rendering.
Scrapfly Screenshot
A Python-based integration utilizing the Scrapfly API to capture high-quality, Javascript-rendered, and geo-targeted screenshots of any web page.
Featured*
