Diffbot is an AI-powered extraction skill that transforms unstructured web content into structured data like articles, products, and organization records.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install diffbot
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install diffbot using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Diffbot?
Diffbot is a sophisticated web data extraction tool that leverages artificial intelligence to automatically identify and extract structured data from any web page. Unlike traditional scrapers that rely on fragile CSS selectors, this Openclaw Skills integration uses machine learning to understand page layouts and content types, making it ideal for gathering product details, news articles, or company information at scale.
By integrating through the Membrane framework, this skill provides a secure and streamlined way for AI agents to interact with the Diffbot API. It handles complex authentication plumbing and credential refreshing automatically, allowing developers to focus on building data-driven logic rather than managing API keys and connection states.
Diffbot Use Cases
- Extracting structured product information, pricing, and availability from e-commerce sites.
- Scraping clean article text, author metadata, and summaries from news outlets and blogs.
- Enriching lead data by searching the Diffbot Knowledge Graph for person and organization details.
- Analyzing sentiment and extracting facts from discussion forums and social threads.
- Automating the collection of job postings or event details from various web sources.
How Diffbot Works
- The user installs the Membrane CLI and authenticates their agent environment to establish a secure communication channel.
- A connection to Diffbot is ensured via the CLI, which handles OAuth or API key management server-side.
- The AI agent searches for specific actions based on natural language intent (e.g., "extract product data").
- The agent executes the selected action, passing necessary parameters like a target URL in JSON format.
- Membrane processes the request through Diffbot's AI engines and returns structured JSON output back to the agent.
Diffbot Setup
First, install the Membrane CLI to manage your Openclaw Skills:
npm install -g @membranehq/cli@latest
Authenticate your agent (replace <agentType> with openclaw):
membrane login --tenant --clientName=openclaw
Create a connection to Diffbot:
membrane connection ensure "https://www.diffbot.com/" --json
Diffbot Data Schema & Taxonomy
The skill organizes web data into specific object types. Each extraction returns a structured JSON object following these schemas:
| Object Type | Primary Fields |
|---|---|
| Article | Headline, Author, Date, Text, Summary, URL |
| Product | Name, Brand, Description, Price, Image URL, Offer URL |
| Organization | Company Name, Employees, Industry, Location |
| Person | Name, Employment History, Education, Social Links |
| Video/Image | Title, Metadata, Dimensions, Duration, Embed URLs |
Diffbot Advanced Features
- Knowledge Graph Search: Perform complex queries using DQL (Diffbot Query Language) to find entities across the web.
- Natural Language Processing: Analyze raw text to extract entities, facts, and sentiment analysis programmatically.
- Direct Proxy Requests: Access any endpoint in the Diffbot API directly through a secure proxy that manages authentication headers and path parameters automatically.
- Long-polling Connections: Use the
--waitflag to handle asynchronous connection building in headless environments. - Automated Page Classification: Use the Analyze Page action to let the AI determine the page type and extract data accordingly without manual configuration.
SKILL.md
Loading
Related Openclaw Skills
flux-image
A high-performance AI image generation and editing skill powered by Black Forest Labs Flux models via AceDataCloud.
arh-pr-workflow
A comprehensive automation skill for managing the lifecycle of feature branches and stacked pull requests using Uber's arh CLI tool.
ThreatBook Intelligence Skills
A powerful integration of ThreatBook online threat intelligence APIs for automated file analysis, reputation querying, and compromise detection.
Kling Video
A professional-grade AI video generation skill that interfaces with the Kuaishou Kling API to transform text and images into cinematic video content.
Conviction.fm
An AI agent skill for competing in daily crypto prediction pools using natural language strategies and automated execution.
tmux-manager
A YAML-driven automation tool for managing complex tmux sessions, windows, and panes with precision and idempotency.
Featured*
