A TypeScript and JavaScript SDK for Scrapfly designed to handle complex web scraping, JavaScript rendering, anti-bot bypass, and AI-driven data extraction.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install scrapfly-sdk-typescript
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install scrapfly-sdk-typescript using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Scrapfly TypeScript SDK?
The Scrapfly TypeScript SDK is a robust library that enables developers to scrape web content directly from Node.js, Deno, Bun, and edge environments like Cloudflare Workers. It provides native support for bypassing advanced anti-bot measures such as Cloudflare and DataDome, ensuring reliable access to web data. By integrating this capability with Openclaw Skills, developers can automate the collection of clean, structured content for downstream processing.
With built-in capabilities for headless browser rendering, screenshot capture, and LLM-friendly markdown formatting, the SDK simplifies the transition from raw web pages to structured information. Its design permits concurrent operations, making it useful for high-volume data collection workflows.
Scrapfly TypeScript SDK Use Cases
- Extracting clean Markdown or HTML content from web pages for ingestion into LLM pipelines.
- Bypassing strict anti-bot protections such as Cloudflare, DataDome, and Kasada.
- Rendering JavaScript-heavy single-page applications (SPAs) before parsing data.
- Capturing programmatic screenshots of target web pages for visual verification.
- Performing concurrent scraping tasks across multiple endpoints.
How Scrapfly TypeScript SDK Works
- Initialization: Initialize the ScrapflyClient using your Scrapfly API key.
- Configuration: Construct a ScrapeConfig specifying target parameters like URL, JavaScript rendering requirements, anti-bot bypass, or output format.
- Execution: Execute the scrape request via client.scrape() or handle multiple parallel tasks using client.concurrentScrape().
- Data Extraction & Output: Access scraped content directly from result.result.content, or use ExtractionConfig for structured, AI-driven data parsing.
Scrapfly TypeScript SDK Setup
Install the Scrapfly SDK package using your preferred package manager:
npm install scrapfly-sdk
For Deno or JSR-supported environments, import directly from the JSR registry:
import { ScrapflyClient, ScrapeConfig } from 'jsr:@scrapfly/scrapfly-sdk';
Ensure your API key is configured in your environment variables:
export SCRAPFLY_API_KEY="your-api-key"
Initialize the client in your code:
import { ScrapflyClient } from 'scrapfly-sdk';
const client = new ScrapflyClient({ key: process.env.SCRAPFLY_API_KEY });
Scrapfly TypeScript SDK Data Schema & Taxonomy
The SDK organizes scraper outputs and configurations systematically. Key classes and their configuration options include:
| Configuration Class | Supported Options | Output / Result Format |
|---|---|---|
ScrapeConfig |
url, asp (boolean), country (ISO country code), proxy_pool, render_js (boolean), wait_for_selector, rendering_wait (ms), format |
result.result.content (contains scraped raw, clean_html, json, markdown, or text data) |
ScreenshotConfig |
url |
Binary image data (shot.image) |
ExtractionConfig |
body (HTML string), content_type, extraction_prompt |
Parsed structured JSON data (data.data) |
Use the format: 'markdown' parameter in your scrape configuration to generate LLM-ready text content for compatibility with Openclaw Skills workflows.
Scrapfly TypeScript SDK Advanced Features
- Anti-Bot Bypass (ASP): Bypasses scraping mitigation tools like Cloudflare and Kasada when
asp: trueis combined with residential proxy pools. - Geographic Targeting: Allows routing requests through specific country proxies (using ISO 3166-1 alpha-2 codes) and custom proxy pools.
- Headless Browser Interactions: Supports JavaScript rendering, waiting for specific DOM selectors, and setting custom rendering delays.
- AI-Driven Data Extraction: Uses ExtractionConfig with custom natural language prompts to automatically structure raw HTML into custom JSON objects.
- Asynchronous Concurrency: Implements
client.concurrentScrapefor parallel processing of multiple scraping targets with minimal overhead.
SKILL.md
Loading
Related Openclaw Skills
Scrapfly Alerting
A threshold-based monitoring integration to watch Scrapfly account metrics and trigger notifications via webhooks, email, or in-app alerts.
Scrapfly Go SDK
An advanced web scraping SDK for Go that enables seamless proxy rotation, anti-bot bypass, JavaScript rendering, and automated data extraction.
Scrapfly Cloud Browser
Automate cloud browsers using the Scrapfly Cloud Browser API with Python Playwright for residential proxy rotation and anti-bot fingerprint management.
StablePay Agent Bootstrap
A dedicated setup skill to bootstrap, validate, and integrate the StablePay MCP server and CLI utility for AI coding agents.
Scrapfly Extraction
An integration for extracting structured data from raw web content, text, or markdown using the Scrapfly Extraction API and AI-driven models.
Scrapfly Scraper
An advanced web scraper leveraging the Scrapfly API to extract web data with anti-bot bypass, proxy rotation, and headless browser rendering.
Featured*
