# Computer Use

Drive the desktop, macOS, or the Windows desktop through WSL, when a task needs a...

- Repository: https://github.com/clacky-ai/openclacky (1209 stars)
- Category: Media
- License: MIT
- Last updated: 2026-10-30
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add clacky-ai/openclacky --skill computer-use -g -a claude-code -y`
Codex: `npx skills add clacky-ai/openclacky --skill computer-use -g -a codex -y`
Cursor: `npx skills add clacky-ai/openclacky --skill computer-use -g -a cursor -y`
Gemini CLI: `npx skills add clacky-ai/openclacky --skill computer-use -g -a gemini-cli -y`

## Description

Drive the desktop, macOS, or the Windows desktop through WSL, when a task needs a native app, dialog, menu bar, file picker or anything the terminal and browser tools cannot reach. Screenshot first, then click and type by coordinates read from that image.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Computer Use (macOS and Windows)

GUI control goes through the script bundled with this skill. The `computer-use` extension is
off by default, so if you are reading this the user has switched it on.

## Platform support

| Host | Backend | Needs |
|---|---|---|
| macOS | CoreGraphics + Quartz events | Screen Recording and Accessibility grants |
| Windows | PowerShell agent launched from inside WSL | WSL running in an interactive desktop session |

On Windows the script only works from a WSL shell on the same machine: it reaches the
Windows desktop through WSL interop, which inherits the logged-on user's session. Run
from a service, an SSH session or anything on session 0 and `doctor` will say so, screenshots come back black and input goes nowhere. This does not work on plain Linux
without Windows, and there is no macOS-side fallback for it.

Everything below is the same on both platforms. Key combos keep the macOS spelling
(`cmd`, `shift`), so `cmd` means the Windows key on Windows.

## When to use it

- Native apps, dialogs, menu bars, installers, file pickers, System Settings.
- Anything on screen that `terminal` cannot reach and `browser` cannot automate.
- For web automation prefer the `browser` tool while it works. If it fails because remote
  debugging is unavailable, fall back here immediately, do not open `chrome://inspect` or run
  the browser-setup skill unless the user asks for it.

## Workflow

1. Bring the target app forward if another window may cover it:

   ```bash
   ruby <skill_dir>/bin/computer.rb activate "WorkBuddy"
   ```

   It needs no extra permission on either platform. If the name does not match (localized
   app names differ), it prints what is actually frontmost, take a screenshot
   to see the real window order instead of guessing.

2. Screenshot, with a labelled grid when you will need to click precise spots:

   ```bash
   ruby <skill_dir>/bin/computer.rb screenshot --grid
   ```

The rest: https://github.com/clacky-ai/openclacky/blob/HEAD/lib/clacky/default_extensions/computer-use/skills/computer-use/SKILL.md