A zero-dependency Python tool for hardware-level mouse and keyboard automation on macOS using CGEvent and AppleScript.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install macos-native-automation
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install macos-native-automation using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is macOS Native Automation?
This skill provides a robust solution for AI agents when traditional browser automation methods like CDP or JavaScript .click() fail. By utilizing the macOS CoreGraphics CGEvent API, it injects events at the Human Interface Device (HID) layer, ensuring that apps—including browsers, Electron apps, and native software—treat the input as a physical mouse action.
This approach is essential for Openclaw Skills that need to interact with security-blocked elements, React dropzones, or native file dialogs that are invisible to standard web drivers. By operating below the application layer, it provides a universal bridge for automating any interface on the macOS desktop.
macOS Native Automation Use Cases
- Handling native file upload dialogs and security prompts that block standard browser automation
- Interacting with React dropzones, canvas elements, and gesture-gated UI components
- Automating complex multi-step workflows across different native macOS applications
- Performing pixel-perfect drag-and-drop operations that require hardware-level precision
- Navigating system-level menus and dialogs using a combination of CGEvent and AppleScript
How macOS Native Automation Works
- The skill captures the target application's window coordinates or uses screen-capture measurements to identify precise pixel locations.
- Using Python's ctypes to interface with the macOS CoreGraphics framework, it generates synthetic mouse events at the HID layer.
- The operating system processes these events as physical inputs, allowing the agent to click, double-click, or drag anywhere on the screen.
- For native UI elements like 'Go to Folder' sheets, the skill utilizes AppleScript via the Accessibility API to handle keyboard input.
- The integrated workflow ensures that the target application is activated and focused before executing time-sensitive automation sequences.
macOS Native Automation Setup
To begin using this skill within your Openclaw Skills environment, follow these steps:
- Ensure you are running macOS with Python 3 installed (no external pip packages are required).
- Grant Accessibility permissions to your terminal, IDE, or the agent runner: navigate to
System Settings->Privacy & Security->Accessibilityand add your application. - Test the installation by retrieving a window position:
python3 scripts/macos_click.py window "Safari"
- Execute a test click at specific screen coordinates:
python3 scripts/macos_click.py click 500 300
macOS Native Automation Data Schema & Taxonomy
The skill manages interaction data through a coordinate-based system and window metadata:
| Attribute | Description | Format |
|---|---|---|
| Coordinates | Absolute screen pixels (logical pixels on Retina displays) | (x, y) |
| Window Metrics | Metadata including x, y offset, width, and height | JSON Object |
| Display Space | Support for multi-monitor setups with extended x/y ranges | Integer Range |
| CLI Commands | Standardized triggers for click, move, and drag actions | String Arguments |
This schema ensures that Openclaw Skills can reliably map browser viewport locations to absolute system coordinates.
macOS Native Automation Advanced Features
- Low-level HID event injection that punches through application-layer security blocks
- Multi-monitor support allowing automation to span across all connected displays
- Seamless integration between CoreGraphics mouse events and AppleScript keyboard navigation
- Dynamic window discovery to find and target specific application instances by name or ID
- Zero-dependency Python implementation utilizing ctypes for high performance and portability
SKILL.md
Loading
Related Openclaw Skills
Fastmail JMAP Email Skill
Give your AI agent full email superpowers to read, search, and send messages using Fastmail JMAP.
Agent Rate Limiter
A zero-dependency Python utility that prevents AI agents from hitting API rate limits through proactive tier-based throttling and automatic exponential backoff.
GoHighLevel Account & App Setup
A comprehensive guide for AI agents to automate GoHighLevel account registration, developer app creation, and API credential management.
ElevenLabs Open Account
A comprehensive guide for developers and creators to set up ElevenLabs accounts and secure API keys for advanced voice AI integrations.
Workspace Audit
A zero-dependency audit suite for OpenClaw workspaces that prevents file drift, secret leaks, and configuration mismatches.
Dynamic UI Skill
A visualization engine for AI agents to render tables, charts, and dashboards as high-quality images using HTML templates.
Featured*
