A specialized scraping and automation skill featuring platform-specific bypass configs and legal compliance recipes for major Chinese websites using Scrapling.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install cn-data-scraper
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install cn-data-scraper using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Chinese Data Scraper (cn-data-scraper)?
Chinese Data Scraper is an essential package in the Openclaw Skills library, designed to bridge the gap between generic web crawling frameworks and the highly sophisticated anti-scraping ecosystems of major Chinese platforms. While general tools solve standard crawling tasks, the Chinese web ecosystem presents distinct hurdles, including multi-layer cookie verification, mandatory login gates, and rapid UI redesigns. This skill equips developer agents with robust, platform-specific bypass configurations for top-tier sites like Baidu, Taobao, Douyin, Zhihu, WeChat, 1688, Xiaohongshu, Weibo, CSDN, and Boss Zhipin.
By incorporating this skill into your AI workflows via Openclaw Skills, you unlock not only high-speed, adaptive selector code snippets but also a comprehensive legal compliance framework. Developers and AI agents can analyze the ethical boundaries of their scraping plans under Chinese laws like the Personal Information Protection Law (PIPL) and Data Security Law, ensuring data collection remains efficient, highly optimized, and legally compliant.
Chinese Data Scraper (cn-data-scraper) Use Cases
- E-Commerce Price Intelligence: Extraction of live product pricing, minimum order quantities, and wholesale supplier details from Taobao, Tmall, and 1688.
- Social Media Trend Tracking: Fetching trending topics, video metadata, likes, comments, and post contents from Douyin, Xiaohongshu, and Weibo.
- Search Result Scraping: Collecting search engine indexing patterns and metadata directly from Baidu's dynamic search pages.
- Technical & Academic Research: Extracting technical blogs and academic discussions from CSDN and public question-and-answer threads from Zhihu.
- Legal Compliance Pre-Flight Check: Evaluating scraping workflows automatically against China's legal boundaries to prevent violations of personal information protection laws.
How Chinese Data Scraper (cn-data-scraper) Works
- Compliance Strategy Check: The user or AI agent queries the skill to verify if the targeted scraping scope adheres to Chinese legal boundaries (PIPL, DSL, Criminal Law Art. 253/285/286).
- Platform Recipe Selection: The skill fetches the specific, pre-optimized configurations for the targeted Chinese website (e.g., cookie structures for Baidu, login endpoints for Taobao).
- Stealth Engine Initialization: The Scrapling framework is initialized with adaptive parameters, deploying anti-fingerprint browsers (like Camoufox) and setting custom HTTP headers.
- Polite Execution: The script makes requests utilizing custom rate-limiting algorithms to mimic real user patterns and bypass security triggers.
- Adaptive Selector Parsing: The target elements are extracted using structural, partial-class, or text-based selectors to guarantee scripts survive frequent website redesigns.
Chinese Data Scraper (cn-data-scraper) Setup
Prerequisites
Install the Scrapling framework with all required headless browser engine integrations:
# Install Scrapling with all features
pip install scrapling[all]
# Or execute a minimal installation
pip install scrapling
Quick-Start Script
Create a basic scraper using a custom platform recipe from Openclaw Skills:
from scrapling import StealthyFetcher
# Baidu search with anti-crawl bypass
page = StealthyFetcher.fetch(
'https://www.baidu.com/s?wd=Openclaw',
headless=True,
network_idle=True,
timeout=30000,
)
# Extract search results using structural selectors
results = page.css('div.c-container')
for result in results:
title = result.css_first('h3 a')
snippet = result.css_first('[class*="content"]')
if title:
print(f"Title: {title.text()}")
Chinese Data Scraper (cn-data-scraper) Data Schema & Taxonomy
The skill provides templates to standardize data extraction models across different niches. Below are the key schemas provided for standard integrations:
E-Commerce Schema
| Field | Description | Target Platforms | Selector Strategy |
|---|---|---|---|
title |
Name of the listed item | Taobao, 1688, JD | span.Title--titleSpan / a.title |
price |
Integer price listed on page | Taobao, 1688, JD | span.Price--priceInt / span.price |
sales / min_order |
Total units sold or wholesale minimums | Taobao, 1688 | span.Sales--sales / span.min-order |
shop / supplier |
Official merchant or company name | Taobao, 1688 | a.ShopName--shopName / a.company-name |
Social Media Schema
| Field | Description | Target Platforms | Selector Strategy |
|---|---|---|---|
author |
Screen name of the publisher | Weibo, Xiaohongshu, Douyin | a.name / span.name / span.author-card-user-name |
content |
Body text or title description | Weibo, Xiaohongshu, Douyin | p.txt / span.note-text / p.title |
likes |
Direct like / reaction count | Weibo, Xiaohongshu, Douyin | Structural element count / .like-wrapper .count |
comments |
Total comment count | Weibo, Xiaohongshu, Douyin | a[action-type="flcomment"] em / CSS fallback |
Chinese Data Scraper (cn-data-scraper) Advanced Features
- Model Context Protocol (MCP) Server Integration: Native tools that let AI agents programmatically request compliant web scraping operations or perform automated legality assessments on a proposed scraping strategy.
- Smart Adaptive Selectors: Scrapling's unique element identification system that tracks and resolves element characteristics, ensuring scripts do not break during routine updates on Chinese sites.
- Pre-configured Platform Rate-Limits: Built-in delay ranges customized per target platform, minimizing blocks (e.g., maintaining 10 requests per minute on Xiaohongshu, 60 requests per minute on WeChat).
- Anti-Copy Overlay Decoupling: Algorithms targeted at CSDN and other portals to programmatically extract technical data past client-side copy protections.
SKILL.md
Loading
Related Openclaw Skills
NAS Dashboard
An alert-first, exception-driven text dashboard for NAS and HomeLab monitoring that delivers surgical error reports and actionable fix commands.
Ready Tools on SciMiner
A dynamic Openclaw skill that discovers, reads, and executes arbitrary SciMiner API tools directly from live Markdown documentation.
Personal Assistant
A local SQLite-powered task management assistant that automates daily routines, OKR tracking, reminders, and performance reports.
Bark Notify
A dedicated skill to send instant Bark push notifications to iOS and macOS devices once AI coding tasks or agent runs are completed.
Agent Memory Cleanup
A lightweight, secure utility to audit, prune, and sanitize long-term AI agent memory files while ensuring safe backups.
Code Audit Agent Skill
A read-only, evidence-backed code and security auditing tool designed for AI agents to discover vulnerabilities, regressions, and design drift without modifying code.
Pentest Findings Report
An automated reporting assistant that structures raw security findings into industry-compliant penetration test reports.
Featured*
