A powerful automation skill designed to control a physical iPhone through macOS iPhone Mirroring, allowing agents to interact with iOS apps directly.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install iphone-use
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install iphone-use using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is iphone-use?
The iphone-use skill is a cutting-edge developer tool designed to drive a real, physical iPhone through the iphone-use daemon using macOS iPhone Mirroring. Unlike API-dependent integrations, this tool utilizes the native mirroring system on macOS to interact with any on-phone application, including banking, Apple Health, and instant messaging apps that completely lack external APIs. It offers robust capabilities to view the screen, send precise touch coordinates, text, and gestures, and extract UI structure, serving as a cornerstone for advanced mobile automation workflows.
By incorporating this skill into Openclaw Skills repositories, developers and AI agents can execute end-to-end device testing, scrape non-web mobile data, and coordinate interactions across physical devices. The skill seamlessly switches between standard Mirroring mode and a dedicated Agent mode powered by WebDriverAgent (WDA) to provide deep, structured visibility into the iOS UI tree for rapid and reliable execution.
iphone-use Use Cases
- Automating iOS applications that do not expose public APIs, such as Apple Health, local banking, or instant messaging platforms.
- Scraping and exporting on-phone data to a connected macOS system (e.g., executing a full Apple Health data export).
- Building automated UI testing suites for iOS apps directly on physical devices using real-world interactions.
- Running multi-step mobile workflows that require physical screen interaction, tapping, scrolling, and keyboard typing.
- Utilizing Openclaw Skills to design self-healing mobile scripts that switch from vision-based execution to structured programmatic scripts.
How iphone-use Works
- Status Verification: The AI agent polls the /agent/status endpoint to verify connectivity and ensure the device is in a drivable state (WDA is running or the mirror shows live content).
- UI Inspection: If WebDriverAgent is active, the agent retrieves the structured UI hierarchy via /agent/elements to target elements programmatically, falling back to /agent/screenshot only when full visual layout processing is required.
- Action Execution: The agent sends targeted inputs to /agent/input (including taps, scrolls, text inputs, long-presses, or system shortcuts) using normalized, resolution-independent coordinates.
- Transition and Verification: The system waits briefly for animations to complete, then recaptures the state via screenshots or UI elements to verify the action succeeded before proceeding to the next step.
iphone-use Setup
Ensure your host Mac is running the iphone-use daemon with iPhone Mirroring connected. You can register the daemon as a LaunchAgent via the provided repository script.
Configure the environment variables:
export PHONE_REMOTE_URL="http://127.0.0.1:44321"
export PHONE_REMOTE_TOKEN="your_daemon_password_or_token"
Verify that the device is drivable by probing the status endpoint:
curl -s -H "Authorization: Bearer $PHONE_REMOTE_TOKEN" "$PHONE_REMOTE_URL/agent/status"
If the skill is outdated or does not match your current system version, update your Openclaw Skills dependencies:
npx skills update -g
iphone-use Data Schema & Taxonomy
The iphone-use skill coordinates data transfer via structured HTTP JSON endpoints. Below are the primary data schemas and coordinates utilized:
Coordinate System
All coordinates are normalized float values spanning from 0.0 to 1.0 relative to the screen dimensions, making interaction scripts entirely resolution-independent.
API Reference
| Endpoint | Method | Response Payload / Behavior |
|---|---|---|
/agent/status |
GET | Status of the daemon, including drivable, mirror_state, viewer_count, and blocking setup states (e.g., warp, trust). |
/agent/elements |
GET | Structured JSON list representing the UI tree, including element kind, label, depth, and values. |
/agent/screenshot |
GET | Raw PNG screen capture of the physical device screen. |
/agent/input |
POST | Accepts action objects such as tap (with label or x/y coordinates), scroll, text, key, and shortcut. |
iphone-use Advanced Features
- WebDriverAgent (WDA) Integration: Unlocks programmatic, text-based UI tree parsing (/agent/elements) which is significantly faster and less resource-intensive than vision-based capture.
- Vision-to-Script Automation: Enables agents to bootstrap workflows visually and freeze successful execution paths into permanent, reusable bash or Python scripts.
- Dual-Mode Mirror & Agent Execution: Seamlessly alternates between the macOS iPhone Mirroring interactive window and dedicated device agent modes to optimize for user presence.
- Clipboard Syncing: Employs advanced fallback pasting mechanisms that preserve and restore the macOS clipboard, bypassing standard simulator keyboard latency issues.
- Smart Interstitial Management: Intelligent handling of paused connections, locked devices, and VPN blocks (such as Cloudflare WARP) with detailed status hints for self-healing runs within Openclaw Skills ecosystem.
SKILL.md
Loading
Related Openclaw Skills
Web to WeChat
An automated workflow that scrapes any web page, structures it using AI, generates and compresses cover art, and publishes it directly into WeChat draft box.
Plan Verify Skill
An automated planning and verification skill optimized for maritime and travel logistics, deployed via CruiseSkillBridge.
Piaoin Invoice
An automated invoice management skill that bridges local directories with the Piaoin PAT API for seamless synchronization, structured tracking, and document uploads.
lygo-alignment-badge
A validation tool to verify LYGO node alignment, run multi-phase demo cycles, and output machine-readable status badges.
cruise-product-info-mcp
An MCP server that delivers comprehensive cruise product details, onboard features, and services to assist users and agents in travel decision-making.
Cruise Product Info
An MCP-compliant tool designed to fetch comprehensive, real-time cruise product details, itinerary data, and service features to power intelligent travel assistants.
Featured*
