A powerful automation suite for controlling macOS UI elements using mouse, keyboard, and system-level scripts.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install mac-control
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install mac-control using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Mac Control?
The Mac Control skill provides a robust interface for AI agents to interact with the macOS desktop environment. By leveraging industry-standard tools like cliclick and AppleScript, this skill enables precise mouse movements, clicks, and keyboard inputs. It is designed to bridge the gap between headless code execution and graphical user interface interaction, allowing Openclaw Skills to perform tasks within native applications and web browsers alike.
This skill handles complex desktop scenarios, including high-resolution Retina display coordinate scaling and window management. Whether you are automating repetitive tasks in Chrome or interacting with system dialogs, Mac Control ensures reliable execution through a screenshot-driven verification workflow. It is an essential component for any Openclaw Skills deployment requiring deep OS integration.
Mac Control Use Cases
- Automating interactions with Chrome extensions and browser toolbars.
- Dismissing system-level dialogs or security prompts that block automated workflows.
- Performing GUI testing on native macOS applications.
- Handling complex multi-window workflows where CLI tools are insufficient.
- Navigating protected login pages like Google OAuth using keyboard-based fallbacks when mouse events are blocked.
How Mac Control Works
- Capture: The skill uses the built-in screencapture tool to take a silent snapshot of the current display state.
- Analysis: The agent analyzes the screenshot to identify UI elements and their specific pixel coordinates.
- Calibration: If using a Retina display, the skill calculates the 2:1 coordinate scaling to ensure click accuracy.
- Execution: It triggers cliclick or osascript to perform mouse actions (click, drag, double-click) or keyboard input.
- Verification: A follow-up screenshot is taken to confirm the UI state has changed as expected, completing the Openclaw Skills feedback loop.
Mac Control Setup
Install the necessary dependencies via Homebrew and grant the required accessibility permissions to your environment:
brew install cliclick imagemagick
# Grant Accessibility permissions in System Settings
# Privacy & Security -> Accessibility -> Add your terminal or /opt/homebrew/bin/node
Verify your display scaling using the provided calibration script:
./scripts/calibrate-cursor.sh
Mac Control Data Schema & Taxonomy
The skill manages UI data through visual snapshots and system queries to ensure Openclaw Skills have high situational awareness:
| Data Element | Storage/Format | Purpose |
|---|---|---|
| UI Snapshots | /tmp/screenshot.png | Temporary image files used for visual analysis and coordinate mapping. |
| Window Bounds | AppleScript Array | Detailed (left, top, right, bottom) coordinates for active application windows. |
| Cursor Position | X, Y Coordinates | Real-time tracking of the system mouse location via cliclick. |
| Process Metadata | String List | Identifiers for running non-background processes to target specific apps. |
Mac Control Advanced Features
- Color Detection: Leverage ImageMagick to find specific UI elements based on hex color codes when standard selectors fail.
- Keyboard Fallbacks: Automated Tab-and-Enter navigation specifically designed for pages that block synthetic mouse events.
- Multi-Window Management: Logic to handle tab listing and window counts in Google Chrome for complex browser automation.
- Visual Verification Workflow: A strict capture-move-capture cycle to ensure mouse actions land on the intended target before execution.
SKILL.md
Loading
Related Openclaw Skills
m3u8-downloader
A high-performance utility for Openclaw Skills that automates the extraction, parallel downloading, and decryption of HLS video streams.
Grok Browser
A sophisticated browser automation tool designed to query Grok AI and extract responses via Openclaw Skills and Chrome relay.
CapMonster Cloud CAPTCHA Solver
An API-driven tool for programmatically bypassing reCAPTCHA, hCaptcha, Turnstile, and image CAPTCHAs during browser automation.
Anemone Browser
A managed headful Chrome browser designed for agents, featuring anti-detection, human-in-the-loop VNC takeover, and multi-session isolation.
Sui Move Test Coverage & Security Auditor
An automated toolkit for Sui Move developers to maximize test coverage and conduct deep security audits of smart contracts.
sui-decompile
A powerful tool for fetching and analyzing decompiled Sui Move smart contract source code directly from block explorers.
Featured*
