# Weavebench Cua Reproduce

Reproduce CUA-Harness experiments on WeaveBench from a GitHub checkout.

- Repository: https://github.com/AMAP-ML/LongHorizon-Harness (1799 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-10-21
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add AMAP-ML/LongHorizon-Harness --skill weavebench-cua-reproduce -g -a claude-code -y`
Codex: `npx skills add AMAP-ML/LongHorizon-Harness --skill weavebench-cua-reproduce -g -a codex -y`
Cursor: `npx skills add AMAP-ML/LongHorizon-Harness --skill weavebench-cua-reproduce -g -a cursor -y`
Gemini CLI: `npx skills add AMAP-ML/LongHorizon-Harness --skill weavebench-cua-reproduce -g -a gemini-cli -y`

## Description

Reproduce CUA-Harness experiments on WeaveBench from a GitHub checkout. Use when the user wants an AI coding agent to set up dependencies, download WeaveBench assets, prepare the 120G VM, configure Qwen/Anthropic-compatible APIs, run smoke tests, launch full or subset evaluations, inspect logs, or summarize scores for this repository.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# WeaveBench CUA-Harness Reproduction

Use this skill to guide an agent through a complete, reproducible WeaveBench CUA-Harness run. The expected repository layout is:

```text
WeaveBench-harness/
  WeaveBench/
  cua-harness/
  skills/weavebench-cua-reproduce/
```

Do not invent alternate launch commands. Prefer the bundled helper script and the project scripts under `WeaveBench/scripts/`.

## Supported Scope

Supported automation:

- Local Docker/KVM WeaveBench evaluation.
- Qwen 3.7-Plus through an Anthropic-compatible endpoint.
- Claude Code execution backend through `cua_harness_claudecode`.
- OpenClaw judge setup and verification.
- Official WeaveBench assets downloaded from the HuggingFace dataset.
- A 120G copy of the official Ubuntu qcow2 plus per-task rootfs growth.

Not automated:

- Cloud VM provisioning.
- Public image-hosting service deployment.
- New model-provider adapters beyond the existing Anthropic-compatible path.
- Non-Claude-Code execution backend reproduction.
- Deleting or pruning existing results.

If the user requests unsupported automation, explain the boundary and provide the closest supported local Docker/KVM path.

## Start With Intake

Before doing setup work on a new machine, run or mentally follow:

```bash
./skills/weavebench-cua-reproduce/scripts/reproduce.sh intake
```

Confirm:

- Linux Docker/KVM is available or installable.
- The user accepts large downloads and creation of `Ubuntu_120G.qcow2`.
- The model endpoint and API key are available.
- Image proxy should be enabled with public upload/show URLs, or disabled because the endpoint accepts large base64 screenshots.
- The desired scope is `doctor`, `smoke`, subset, or full 114-task evaluation.
- Long-running/full evaluation is allowed, especially if other experiments may already be active.

## Workflow

The rest: https://github.com/AMAP-ML/LongHorizon-Harness/blob/HEAD/eval/WeaveBench-harness/skills/weavebench-cua-reproduce/SKILL.md