A comprehensive toolkit for Windows desktop automation including screen capture, OCR text extraction, and image-based UI element discovery.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install windows-skills
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install windows-skills using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Windows Desktop Automation Skill?
The Windows Desktop Automation skill is a robust extension for Openclaw Skills designed to bridge the gap between AI agents and the Windows graphical user interface. By combining powerful libraries like MSS for high-performance screen capture, PyTesseract for optical character recognition, and OpenCV for template matching, this skill enables agents to see and interact with any desktop application. It provides a developer-friendly interface to programmatically handle visual data, making it an essential component for building robotic process automation (RPA) workflows within the Openclaw Skills ecosystem.
Windows Desktop Automation Skill Use Cases
- Automating data extraction from legacy desktop applications that lack native APIs.
- Capturing specific window regions for visual monitoring or automated documentation.
- Identifying and clicking UI elements like buttons or icons based on visual templates.
- Implementing multi-language text recognition from screenshots to drive logic in AI workflows.
How Windows Desktop Automation Skill Works
- The skill utilizes the mss library to capture high-resolution screenshots of the full screen, specific regions, or target windows by title.
- Captured images are passed to the OCR module where Tesseract processes the visual data into structured text strings.
- For UI interaction, the image location module uses OpenCV to scan the screen for matches against a provided template image.
- Once an element is located, the system returns precise (x, y) coordinates and confidence scores.
- These coordinates can then be used by libraries like PyAutoGUI to perform physical interactions like clicks or scrolls.
Windows Desktop Automation Skill Setup
To get started with this skill in your Openclaw Skills environment, install the necessary Python dependencies:
pip install mss pytesseract pillow pyautogui opencv-python numpy
Note: This skill requires Tesseract OCR to be installed on your system and added to your system PATH for text recognition capabilities.
Windows Desktop Automation Skill Data Schema & Taxonomy
The skill organizes its functionality across several specialized scripts and utilities:
| Script | Description | Primary Output |
|---|---|---|
screenshot.py |
Handles all screen capture logic | Image files (PNG, JPG) |
ocr.py |
Manages text extraction and language processing | String / Text data |
image_locate.py |
Performs template matching and coordinate discovery | (x, y) Coordinates |
helpers.py |
Shared utility functions for image processing | Internal data objects |
Windows Desktop Automation Skill Advanced Features
- Multi-language support for OCR, enabling simultaneous extraction of English and Simplified Chinese text.
- Window-specific targeting allowing the agent to capture content even when windows are partially obscured.
- High-precision image matching with confidence thresholds to prevent false positives in UI automation.
- Seamless integration with PyAutoGUI for end-to-end desktop control within Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
Linear
A powerful command-line interface for Linear that integrates issue tracking and project management directly into your development workflow.
Aavegotchi Renderer Bypass
An automated workflow for generating and downloading Aavegotchi 2D and 3D assets using on-chain data and the official renderer API.
Aavegotchi Gotchiverse Player Operations
A specialized toolset for managing player workflows, crafting, and parcel development within the Aavegotchi Gotchiverse on the Base mainnet.
Aavegotchi GBM Auction Skill
A high-security interface for managing Aavegotchi GBM auctions on the Base mainnet using subgraph discovery and Foundry-based execution.
Civic Nexus MCP Bridge
A versatile bridge connecting AI agents to over 100 external integrations through the Model Context Protocol.
Civic MCP Bridge
A comprehensive integration bridge connecting Openclaw to over 100 external services via the Civic Model Context Protocol.
Featured*
