A robust Xiaohongshu search, note reader, and user profile scraping skill integrated with the agent-browser JS environment.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install xhs-cli
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install xhs-cli using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Xiaohongshu Content Harvester & Scraper?
The Xiaohongshu Search & Scraping skill is a powerful crawler designed for Openclaw Skills that enables automated scraping, searching, and metadata extraction from the Xiaohongshu (Red) platform. By leveraging the official agent-browser JS environment, this skill automatically handles sophisticated X-s signature generation and bypasses standard web anti-scraping mechanisms.
It is one of the most robust Openclaw Skills for content curation, competitive intelligence, and social media analytics. By utilizing a valid user session cookie, the skill allows seamless collection of notes, user feeds, and comments without getting trapped in endless verification loops.
Xiaohongshu Content Harvester & Scraper Use Cases
- Conduct competitive analysis and market research on Xiaohongshu.
- Automate content curation by fetching popular notes under specific trending topics.
- Track creator profile analytics and download videos and images from specific feeds.
- Build rich datasets for social media trend analysis with Openclaw Skills workflows.
- Batch retrieve comments and sentiment data from targeted notes.
How Xiaohongshu Content Harvester & Scraper Works
- Session Verification: The skill validates your login cookies using
xhs-keepalive.py checkto ensure the session is active. - Target Identification: Based on user parameters, it either searches for a keyword or queries a specific user ID.
- Optimal Path Selection: To evade captchas, the skill defaults to direct user-profile scanning (Bucket 4), completely bypassing risky search buckets.
- Data Retrieval: It parses DOM elements to extract note items, source tokens, and detailed engagement stats.
- Structuring Results: Collected raw data and metadata are saved locally and a clean Markdown report is generated.
Xiaohongshu Content Harvester & Scraper Setup
First-time setup is straightforward but requires extracting cookies from an active browser session:
- Log in to
xiaohongshu.comin your browser. - Open Developer Tools (F12), go to the Console, and run
copy(document.cookie). - Paste the copied cookie string into the raw cookies file:
cat > $SKILL/data/cookies-raw.txt <<'EOF'
a1=xxx; web_session=xxx; id_token=xxx; ...
EOF
chmod 600 $SKILL/data/cookies-raw.txt
- Convert and validate the cookies using the built-in keepalive script:
python3 $SKILL/xhs-keepalive.py inject
python3 $SKILL/xhs-keepalive.py check
Xiaohongshu Content Harvester & Scraper Data Schema & Taxonomy
Data is organized in structured directories under the $SKILL/data/harvests/ workspace:
| Path | Format | Description |
|---|---|---|
user-<uid>-<timestamp>/ |
Directory | Root folder containing the specific harvesting run metadata. |
REPORT.md |
Markdown | Clean, human-readable summary of the scraped notes and comment sections. |
user.json |
JSON | Holds profile metadata and the list of discovered notes with their tokens. |
notes/ |
Directory | Directory containing detailed raw content for each note. |
notes/n01-*.json |
JSON | Note detail document including raw text, likes, and comment objects. |
Xiaohongshu Content Harvester & Scraper Advanced Features
- Smart Bypass Bucket Routing: Prioritizes direct user profile page paths (Bucket 4) to ensure 100% success rates even when keyword searches are captcha-blocked.
- Strict Error Safeguards: Fails fast (Error 300012) on captcha limits instead of auto-retrying, protecting your IP address from getting flagged.
- Native Proxy Support: Supports standard
http_proxyintegration for scaled enterprise workloads. - Automated Domestic Guardrails: Includes suffix parsing (
strip_author_suffix) and precise DOM handling to ensure flawless extraction across multiple Openclaw Skills environments.
SKILL.md
Loading
Related Openclaw Skills
Anti-Slop Frontend Skill
An opinionated frontend design skill designed to eliminate generic LLM aesthetic defaults and ship unique, production-grade landing pages, portfolios, and redesigns.
Dynamics 365 CRM
An advanced integration skill that empowers users to manage Microsoft Dynamics 365 CRM records, accounts, contacts, leads, and sales activities directly from their chat client.
Zyte API Web Scraping
An AI-powered web scraping skill for extracting structured content from dynamic websites using proxy rotation and browser rendering.
OneNote Notes
An advanced AI-powered integration that enables seamless management of Microsoft OneNote notebooks, sections, and pages directly within your AI coding agent environment.
EasyLink EasyDoc Mask
An automated AI agent skill to securely redact and mask sensitive personal or medical data in documents using the EasyLink asynchronous masking API.
EasyLink EasyDoc Extract
An asynchronous document extraction skill for AI agents to retrieve structured fields, bounding boxes, and specific key-value pairs from complex files.
Featured*
