An automated tool to scrape and AI-analyze research papers from shitjournal.org using headless browser technology.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install shit-journal-scraper
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install shit-journal-scraper using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Shit Journal Scraper?
Shit Journal Scraper is a specialized automation tool designed to bridge the gap between academic publications and AI-driven insights. Built for the Openclaw Skills ecosystem, it leverages Playwright to handle Single Page Applications (SPAs) and dynamic content rendering. By simulating real user interactions, it bypasses Client-Side Rendering (CSR) limitations to fetch deep data from academic repositories, making it an essential asset for researchers and developers who need structured data from modern web architectures.
Shit Journal Scraper Use Cases
- Automated monitoring of new preprints and research papers on specialized academic journals.
- Generating AI-summarized insights from long research abstracts for quick consumption.
- Building a local knowledge base of academic metadata including titles, DOIs, and core findings.
- Integrating scholarly data into custom Openclaw Skills workflows for competitive analysis or academic review.
How Shit Journal Scraper Works
- Browser Initialization: The skill launches a headless Chromium instance via Playwright to ensure full JavaScript execution.
- Dynamic Page Fetching: It navigates to the target site and waits for the SPA to finish rendering the DOM.
- DOM Parsing: Using JSDOM, the tool builds a document tree to locate specific nodes using precise CSS selectors.
- Metadata Extraction: It extracts key fields such as titles, abstracts, and DOI identifiers from the rendered HTML.
- AI Processing: The system optionally passes extracted text to an LLM to distill core viewpoints and technical summaries.
- Structured Output: The final data is cleaned and returned as a standardized JSON object for further automation.
Shit Journal Scraper Setup
1. Install Dependencies
npm install playwright jsdom
npx playwright install chromium
2. Execution
# Execute the scraping and analysis task
node index.js
Shit Journal Scraper Data Schema & Taxonomy
The skill outputs a structured JSON array containing extracted article details:
| Field | Type | Description |
|---|---|---|
| title | string | The extracted title of the research paper |
| abstract | string | The full summary text found on the page |
| doi | string | The unique Digital Object Identifier (DOI) for the paper |
It filters out entries missing critical titles or abstracts to ensure data quality within the Openclaw Skills environment.
Shit Journal Scraper Advanced Features
- SPA-ready rendering using Playwright Chromium to handle complex client-side applications.
- Intelligent AI dismantling that transforms raw abstracts into actionable core viewpoints.
- Robust error handling that returns standardized JSON error objects instead of crashing.
- Self-adaptive environment management that automatically handles browser driver dependencies.
SKILL.md
Loading
Related Openclaw Skills
Personal Growth Coach
An AI-driven cognitive trainer that generates open-ended exercises based on world-class thinking frameworks to sharpen your communication and analytical skills.
Auto Video Creator: XLXAI Video Generation
A high-performance AI video generation tool leveraging the XLXAI Sora2 API for professional text-to-video and image-to-video creation.
Shang Tsung
A bash-based persistent memory and identity continuity system for AI agents that ensures session-to-session lineage and operational context.
Video Generation MiniMax
A professional-grade video generation skill utilizing the MiniMax API to create high-fidelity videos from text, images, or subject references.
Check Transaction
A specialized tool for retrieving and formatting human-readable blockchain transaction data from the AOX API.
Data Analysis
A professional-grade data analysis tool designed to interpret datasets, assess quality, and generate structured insights using business-centric logic.
Featured*
