A comprehensive automation skill for orchestrating complex workflows across web browsers and native desktop applications.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install web-desktop-automation
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install web-desktop-automation using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Web + Desktop Automation?
The Web + Desktop Automation skill provides a unified framework for AI agents to interact with both web-based interfaces and local operating system environments. By leveraging this addition to the Openclaw Skills ecosystem, developers can build agents capable of navigating websites, performing data extraction, and transitioning seamlessly into local desktop applications for file processing or system-level tasks.
This skill prioritizes reliability by enforcing a hierarchy of execution: using stable browser selectors where possible and falling back to coordinate-based desktop interaction only when necessary. It is designed to bridge the gap between cloud-based services and the local workstation, making it an essential tool for cross-platform RPA and complex developer workflows.
Web + Desktop Automation Use Cases
- Automating multi-step web workflows such as form filling, login sequences, and data scraping.
- Orchestrating tasks that require downloading a file from a portal and immediately processing it in a native desktop app.
- Performing UI testing across both web browsers and electron-based or native desktop applications.
- Syncing data between legacy desktop software and modern web-based APIs or dashboards.
- Handling complex file uploads that require local file system navigation and window switching.
How Web + Desktop Automation Works
- Task Classification: The agent analyzes the request to determine if it requires browser-only, desktop-only, or mixed-mode execution.
- Browser Phase: If web interaction is needed, the skill utilizes Playwright-based templates to navigate, read content, or interact with DOM elements.
- Desktop Phase: For native app interactions, the skill switches context to OS-level control using window-aware methods or coordinate-based inputs.
- Verification: The skill verifies the UI state after each phase to ensure the transition between web and desktop environments was successful.
- Recovery: In case of failure, the agent re-locates elements or windows and attempts more stable interaction methods before requesting user intervention.
Web + Desktop Automation Setup
To get started with this skill within the Openclaw Skills framework, ensure you have the necessary drivers and dependencies installed.
# Install the core dependencies
pip install -r requirements.txt
# Initialize the automation runners
python scripts/browser_runner.py --setup
python scripts/desktop_runner.py --check-env
Refer to references/dependencies.md for specific OS-level requirements (like X11 or Wayland configurations on Linux).
Web + Desktop Automation Data Schema & Taxonomy
The skill maintains a structured directory to organize logic, templates, and reference documentation:
| Directory / File | Description |
|---|---|
scripts/ |
Contains the execution engines: browser_runner.py, desktop_runner.py, and mixed_orchestrator.py. |
references/ |
Stores pattern libraries for browser-workflows.md and desktop-workflows.md. |
requirements.txt |
Lists the minimal Python package set (e.g., Playwright, PyAutoGUI). |
mixed-example.md |
A blueprint for download-edit-upload workflows. |
Web + Desktop Automation Advanced Features
- Mixed Workflow Orchestration: Seamlessly hand off state between browser sessions and desktop applications using the mixed_orchestrator script.
- State Capture: Automatically capture the UI state during long-running or fragile tasks to allow for smarter recovery and logging.
- Modular Templates: Extensible Python templates for Playwright and PyAutoGUI that can be customized for specific enterprise software.
- Context Awareness: Intelligent window-switching logic that identifies applications by title or process rather than just screen coordinates.
SKILL.md
Loading
Related Openclaw Skills
NocoDB v3 CLI
A feature-complete command-line interface for interacting with the NocoDB v3 REST API to manage databases, records, and team collaboration.
AIKA GPS Skill
A specialized GPS tracking skill that integrates AIKA system data into Openclaw to monitor field technicians and calculate precise service ETAs.
Trending Casserole Guide (Wanghong Shaoguo)
A specialized assistant for discovering viral casserole restaurants, planning dining routes, and accessing insider foodie guides.
Article Writer
An automated agent workflow that researches technical topics, generates structured Markdown articles using sub-agents, and publishes directly to WeChat Official Accounts.
Daoist Culture & Chen Lang Profile
A dual-purpose knowledge skill providing deep insights into Daoist philosophy and the professional Openclaw Skills ecosystem developed by Chen Lang.
Ruankao Questions
An expert-level search and retrieval skill for Chinese Advanced Ruankao professional certification exam questions and detailed solutions.
Featured*
