# Jev Desktop

Drive a desktop application from one plain-language goal, or resolve one intent at a...

- Repository: https://github.com/vercel-labs/agent-desktop (6655 stars)
- Category: Design
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add vercel-labs/agent-desktop --skill jev-desktop -g -a claude-code -y`
Codex: `npx skills add vercel-labs/agent-desktop --skill jev-desktop -g -a codex -y`
Cursor: `npx skills add vercel-labs/agent-desktop --skill jev-desktop -g -a cursor -y`
Gemini CLI: `npx skills add vercel-labs/agent-desktop --skill jev-desktop -g -a gemini-cli -y`

## Description

Drive a desktop application from one plain-language goal, or resolve one intent at a time, without ever reading the accessibility tree. Use when automating a desktop app and you want to keep a 150-element JSON tree out of your context. Hand `run.mjs` a whole goal and it observes, decides and acts until the goal is met; hand `act.mjs` a single step when you want to keep the plan yourself. Covers which element, which operation, when to look deeper, and when to stop. Needs a clone of the agent-desktop repository and a TYPESAFE_API_KEY.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# jev-desktop

This skill requires a clone of the agent-desktop repository (the scripts live in `scripts/jev/` and ship in no installed package) and a `TYPESAFE_API_KEY`.

Two entry points over the same screen reading. `run.mjs` takes a goal and drives
until it is met. `act.mjs` takes one step and hands the plan back to you. Neither
puts the tree in your context.

## The loop

```sh
node scripts/jev/run.mjs --app Finder "open the Applications folder"
node scripts/jev/run.mjs --app TextEdit --cursor --text "A line to write." "write the supplied sentence into the document"
```

Each turn reads the screen, asks for one operation and a target for that
operation in the same request, carries it out, and reads again. It prints one
JSON line per turn and one when it stops (shown trimmed).

```json
{"turn":{"step":1,"operation":"CLICK","target":"treeitem \"Applications\"",
         "confidence":1,"ok":true,"delivery":"delivered_verified","changed":true}}
{"stop":"done","confidence":0.96}
```

The operations are `CLICK`, `TYPE_TEXT`, `CHECK`, `UNCHECK`, `EXPAND`,
`COLLAPSE`, `SCROLL`, `DRILL`, `WIDEN`, `WAIT`, `DONE` and `BLOCKED`. Only the
ones something on screen can receive are offered: `DRILL` for a region that
reports it holds more, `WIDEN` only after a `DRILL`. `SCROLL` always scrolls
down. `delivery` is `waited` for `WAIT` and `looked` for `DRILL` and `WIDEN`.

**The target is asked once per operation.** `click_target` is chosen from the
elements that advertise `Click`, `type_text_target` from those that advertise
`SetValue`, and so on. Only the head matching the chosen operation is read, so a
target can never be incompatible with the verb that acts on it.

**Looking is an operation.** A window is read with `--skeleton` first, so a dense
document costs the same first look as a small panel. A region that was cut off
reports how much it holds, and `DRILL` pins it as the root for later turns.
`WIDEN` gives the whole window back.

The rest: https://github.com/vercel-labs/agent-desktop/blob/HEAD/skills/jev-desktop/SKILL.md