Skills

Host Computer Use Macos

Backend-specific macOS guidance for `computer_use_remote`.

Install

npx skills add agent0ai/agent-zero --skill host-computer-use-macos -g -a claude-code -y
Repository
agent0ai/agent-zero19.6k
Category
Design
Updated
3d ago
License
See repository
Context cost
Lightabout 1,151 tokens
Runs scripts
No, instructions only

What it does

Backend-specific macOS guidance for `computer_use_remote`. Load when a tool result reports backend_family/backend_id `macos` or Accessibility-tree features. Covers AX structural targeting, macOS window actions, and screenshot verification.

Host Computer Use - macOS

Use this after host-computer-use when the connected A0 CLI reports the macOS computer-use backend.

Do not use this skill for Linux, Windows, Xpra, Docker, or browser-only tasks. If the backend is not macOS or does not advertise Accessibility-tree support, skip AX actions and follow the generic host computer-use rules.

macOS AX Targeting

macOS backends can advertise structural Accessibility features:

  • accessibility-tree-snapshot
  • accessibility-structural-targeting
  • accessibility-element-click

When these features are present, prefer structural targeting over pixel clicks for named controls such as buttons, menu items, text fields, sheets, alerts, toolbar items, and sidebar rows.

If the backend also advertises native-window-list, window-state, element-index-targeting, or background-dispatch, prefer the generic background loop from host-computer-use: list_windows -> get_window_state -> element_action. If those features are absent, use the AX snapshot/action flow below.

With app-scoped-semantic-targeting, element_action also accepts top-level pid or window_id to scope semantic matching, or target.app_name / target.bundle_id to select one app. Use identifiers from list_windows; app names and bundle IDs match exactly, ignoring case. Without this feature, use get_window_state and its top-level element_index for another app/window.

For example, replace the app identifier and control title with observed values:

{
  "tool_name": "computer_use_remote",
  "tool_args": {
    "action": "element_action",
    "target": {
      "bundle_id": "com.example.editor",
      "role": "AXButton",
      "title": "Save"
    },
    "operation": "press",
    "dispatch": "background"
  }
}

Use ax_snapshot to inspect the frontmost app's bounded Accessibility tree:

Read the whole SKILL.md on GitHub

More in Design