A specialized Python-based scraper designed to aggregate market intelligence, pricing, and supply chain data for the Sichuan pepper and prickly ash oil industry.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install pepper-oil-scraper
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install pepper-oil-scraper using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is pepper-oil-scraper?
The pepper-oil-scraper is a robust data collection engine within the Openclaw Skills ecosystem, specifically engineered to navigate the complexities of the Sichuan pepper and prickly ash oil supply chain. It integrates over 20 specialized adapters to pull multi-dimensional data, ranging from real-time wholesale prices to global market forecasts.
By utilizing this skill, researchers and businesses can automate the gathering of enterprise financial reports, government policy updates, and import/export statistics. It serves as both a high-precision tool for the seasoning industry and a versatile template for broader agricultural product chain scraping projects.
pepper-oil-scraper Use Cases
- Monitoring real-time wholesale and historical price trends for prickly ash and pepper oil across major Chinese trading platforms.
- Aggregating industry research reports and market size forecasts from top-tier consulting firms.
- Tracking enterprise financial health and annual reports for major industry players like Tenwow or Yihai.
- Collecting government agricultural policies, planting area data, and national standards for food safety.
- Analyzing global market demand and international trade dynamics for Sichuan Pepper Oil.
How pepper-oil-scraper Works
- The user triggers the skill by requesting data related to pepper, prickly ash oil, or seasoning supply chains.
- The main_crawler.py script initializes and loads site-specific configurations from targets.json.
- Based on the requested category (Price, Market, Company, etc.), the system activates the corresponding specialized adapters.
- The base_scraper.py handles low-level requests, implementing anti-scraping measures like randomized delays and UA rotation.
- Collected raw data is passed through data_cleaner.py for normalization of units (e.g., converting prices to Yuan/kg).
- The processed data is structured into a unified schema and can be exported as a comprehensive Excel report via export_report.py.
pepper-oil-scraper Setup
To get started with this skill in your environment, ensure Python 3 is installed and run the following commands:
# Install required Python dependencies
pip install requests beautifulsoup4 lxml pandas openpyxl aiohttp fake-useragent --break-system-packages
# Run a full collection across all sources
python scripts/main_crawler.py --all --output /home/claude/pepper_data/
# Or collect specific categories, such as market reports
python scripts/main_crawler.py --category market
pepper-oil-scraper Data Schema & Taxonomy
The skill enforces strict data standardization across all sources to ensure analytical consistency. Every record includes metadata for traceability.
| Attribute | Standardized Unit | Description |
|---|---|---|
| Price | Yuan/KG | Wholesale or purchase price |
| Area | 10,000 Mu | Planting or cultivation area |
| Output | 10,000 Tons | Production volume |
| Currency | 100 Million Yuan | Enterprise revenue or market value |
| Metadata | source_url, crawl_time | Provenance and timestamp |
Files are organized within a structured directory for easy management: config/ for targets, scripts/adapters/ for source logic, and templates/ for reporting.
pepper-oil-scraper Advanced Features
- Built-in Anti-Scraping Suite: Features random delays (2-5s), fake-useragent rotation, and exponential backoff (up to 60s) for 403/429 errors.
- Modular Adapter Architecture: Easily extendable to new websites by adding custom adapters to the registration system.
- Multi-Category Filtering: Support for targeted scraping of price, market, company, government, media, or global data pools.
- Data Normalization Pipeline: Automatically converts disparate industry units into a standardized format for immediate analysis.
- Playwright Integration: Optional support for JavaScript-heavy websites requiring browser rendering.
SKILL.md
Loading
Related Openclaw Skills
3-Body Movement Simulator
A sophisticated guide and computational framework for architecting numerical simulators to solve the chaotic three-body gravitational problem.
ASCII V-Graph Generator
A specialized tool for generating mathematically precise V-shaped ASCII art and text-based graphs for terminal interfaces.
Smart Skill Advisor
A sophisticated solutions architect for your AI agent that transforms complex user goals into high-efficiency, vetted skill stacks.
wechat-to-md
A high-performance Python utility and MCP server designed to convert WeChat Official Account articles into clean Markdown files with locally hosted images.
DeepSync
An AI-powered matching engine that connects people based on core psychological traits and decision-making styles rather than superficial interests.
HPC ElmerFEM
A specialized agent skill for building, debugging, and automating complex ElmerFEM multiphysics simulation workflows and HPC execution.
Featured*
