A PowerShell-powered skill for programmatically controlling mouse, keyboard, and window states on Windows desktops.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install windows-ui-automation
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install windows-ui-automation using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Windows UI Automation?
The Windows UI Automation skill provides a powerful interface for AI agents to interact directly with the Windows graphical user interface. By utilizing native PowerShell scripts, this skill enables precise simulation of user inputs, including cursor movement, clicking, and typing into desktop applications that lack traditional APIs. Integrating this into your Openclaw Skills library allows for the automation of legacy software, system-level tasks, and complex desktop workflows that require human-like interaction with the OS.
Windows UI Automation Use Cases
- Automating data entry and navigation within legacy desktop applications.
- Performing end-to-end UI testing for Windows-based software.
- Managing desktop environments by focusing, minimizing, or maximizing specific windows.
- Creating automated demonstrations or tutorials by simulating mouse and keyboard actions.
How Windows UI Automation Works
- The AI agent identifies the specific UI action required, such as clicking a button at coordinate (X, Y) or typing a string.
- The skill invokes the appropriate PowerShell script, such as mouse_control.ps1.txt or keyboard_control.ps1.txt, with the required arguments.
- The script executes native Windows API calls via PowerShell to move the cursor, click, or send keystrokes.
- Optional verification steps, such as taking a screenshot or checking window focus, ensure the action was successful within the Openclaw Skills environment.
Windows UI Automation Setup
To use this skill, ensure your environment is running Windows with PowerShell enabled. No additional external dependencies are required as it uses native scripts.
# Navigate to the skill directory to verify script availability
cd skills/windows-ui-automation
# Ensure execution policy allows running local scripts if prompted
# powershell Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Windows UI Automation Data Schema & Taxonomy
The skill organizes its automation logic through specific script files and coordinate systems:
| Component | Description | Inputs |
|---|---|---|
| mouse_control.ps1.txt | Manages cursor and click actions | -Action (move, click, rightclick), -X, -Y |
| keyboard_control.ps1.txt | Manages text and special keys | -Text (string), -Key (special key codes) |
| Window Management | Focuses applications via COM | Window Title (string) |
| Coordinates | Monitor mapping | 0,0 (Top-Left primary monitor) |
Windows UI Automation Advanced Features
- Support for complex keyboard chords and special keys like {ENTER}, {TAB}, and {ALT}.
- programmatic window activation using WScript.Shell COM objects for reliable application switching.
- Integration with visual verification workflows to confirm UI states post-action.
- Extensible script architecture allowing for custom PowerShell-based desktop interactions within Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
Trading Memory Management Skill
A specialized memory management system for PAI trading agents that standardizes decision logs, tracks performance, and automates strategy refinement.
HK_LEETF_Trading
A professional AI-driven skill for analyzing Hong Kong leveraged ETF holdings, rebalancing windows, and arbitrage opportunities.
Heartbeat Task Automator
A robust scheduling and execution engine for managing recurring AI agent tasks, trading logic, and system maintenance.
Capability Evolver
A powerful meta-skill that empowers AI agents to autonomously analyze history, self-repair, and evolve their capabilities.
Restaurant Evaluator
A professional evaluation system for catering entrepreneurs to scientifically assess site selection, calculate payback periods, and identify business risks.
Telegram Topic Rename
An automated utility for renaming Telegram forum topics and updating visual icons using the Telegram Bot API.
Featured*
