A specialized skill enabling AI agents to control Windows applications through UI element scanning, clicking, and text input.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install windows-ui-controller
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install windows-ui-controller using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Windows UI Controller?
The windows-ui-controller is a robust automation bridge designed for Windows environments. It empowers AI agents to interact with graphical user interfaces (GUI) of popular software like WeChat, QQ, and NetEase Cloud by leveraging the pywinauto library. This tool is essential for developers using Openclaw Skills who need to automate workflows within native desktop applications that lack traditional APIs.
By providing a structured way to scan buttons, input fields, and menus, this skill transforms the AI from a text-based assistant into a functional desktop operator. It emphasizes reliability through mandatory step-verification, ensuring that every click and keystroke is accurately registered before moving to the next task.
Windows UI Controller Use Cases
- Automating message delivery and contact management in desktop chat clients.
- Streamlining file uploads and downloads in cloud storage applications.
- Performing repetitive data entry into legacy Windows enterprise software.
- Controlling media playback or settings within native Windows utility apps.
- Orchestrating cross-application workflows that involve desktop-only tools.
How Windows UI Controller Works
- The AI initializes the skill and scans the target Windows application to build a map of available UI controls.
- Specific elements are identified using intelligent recognition logic to distinguish between similar controls like search bars and message inputs.
- The skill executes the requested action, such as clicking a button or entering a string of text.
- A verification scan is performed immediately after the action to confirm the UI has updated correctly.
- In the event of a failure, the system captures visual data to diagnose the state of the application and facilitate recovery.
Windows UI Controller Setup
Ensure you are running Windows 10 or 11 with Python 3.8+ installed. To get started with this component of your Openclaw Skills library, follow these steps:
Install via pip for online environments:
pip install pywinauto
For offline or air-gapped environments, use the pre-packaged dependencies included in the skill folder:
cd dependencies
pip install --no-index --find-links=. pywinauto
Windows UI Controller Data Schema & Taxonomy
| Component | Details |
|---|---|
| Skill Definition | SKILL.md containing metadata and OS requirements. |
| Documentation | README.md providing extensive API references and best practices. |
| Dependencies | Local directory containing .whl files for pywinauto, six, comtypes, and pywin32. |
| Target OS | Windows 10/11 (AMD64 architecture recommended for pywin32). |
Windows UI Controller Advanced Features
- Step-by-step verification protocols to ensure high-reliability automation sequences.
- Intelligent caching of UI control locations to optimize speed for frequently used applications.
- Visual assistance modules that leverage screenshots for error analysis and debugging.
- Offline-first dependency management for secure or restricted environment deployments.
- Support for complex UI hierarchies including nested menus and multi-window applications.
SKILL.md
Loading
Related Openclaw Skills
AI Travel Skill Package
A comprehensive AI-driven travel suite providing end-to-end services from intelligent itinerary generation to visa and packing assistance.
AI Intelligent Travel Assistant Suite
A comprehensive AI travel assistant suite providing one-stop services for itinerary planning, destination guides, budget management, and visa assistance.
AI Travel Skill Package
A comprehensive AI-powered suite for end-to-end travel planning, destination research, budgeting, and preparation.
AI Travel Skill Package
A comprehensive AI-driven travel assistant suite that handles everything from itinerary generation and budget calculation to visa guidance and packing lists.
AI Travel Skill Package
A comprehensive AI-powered travel assistant for itinerary planning, budgeting, visa guidance, and smart packing.
Token Safety Checker
A security auditing utility that scans Openclaw configuration files to identify and privatize plaintext secrets via environment variables.