Computer Use
Drive the desktop, macOS, or the Windows desktop through WSL, when a task needs a...
Install
npx skills add clacky-ai/openclacky --skill computer-use -g -a claude-code -y- Repository
- clacky-ai/openclacky1.2k
- Category
- Media
- Updated
- 5d ago
- License
- MIT
- Context cost
- Mediumabout 1,617 tokens
- Runs scripts
- Yes, review before use
What it does
Drive the desktop, macOS, or the Windows desktop through WSL, when a task needs a native app, dialog, menu bar, file picker or anything the terminal and browser tools cannot reach. Screenshot first, then click and type by coordinates read from that image.
Computer Use (macOS and Windows)
GUI control goes through the script bundled with this skill. The computer-use extension is
off by default, so if you are reading this the user has switched it on.
Platform support
| Host | Backend | Needs |
|---|---|---|
| macOS | CoreGraphics + Quartz events | Screen Recording and Accessibility grants |
| Windows | PowerShell agent launched from inside WSL | WSL running in an interactive desktop session |
On Windows the script only works from a WSL shell on the same machine: it reaches the
Windows desktop through WSL interop, which inherits the logged-on user's session. Run
from a service, an SSH session or anything on session 0 and doctor will say so, screenshots come back black and input goes nowhere. This does not work on plain Linux
without Windows, and there is no macOS-side fallback for it.
Everything below is the same on both platforms. Key combos keep the macOS spelling
(cmd, shift), so cmd means the Windows key on Windows.
When to use it
- Native apps, dialogs, menu bars, installers, file pickers, System Settings.
- Anything on screen that
terminalcannot reach andbrowsercannot automate. - For web automation prefer the
browsertool while it works. If it fails because remote debugging is unavailable, fall back here immediately, do not openchrome://inspector run the browser-setup skill unless the user asks for it.
Workflow
-
Bring the target app forward if another window may cover it:
ruby <skill_dir>/bin/computer.rb activate "WorkBuddy"It needs no extra permission on either platform. If the name does not match (localized app names differ), it prints what is actually frontmost, take a screenshot to see the real window order instead of guessing.
-
Screenshot, with a labelled grid when you will need to click precise spots:
ruby <skill_dir>/bin/computer.rb screenshot --grid
