# Monitor Experiment

Monitor running experiments, check progress, collect results.

- Repository: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep (18443 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill monitor-experiment -g -a claude-code -y`
Codex: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill monitor-experiment -g -a codex -y`
Cursor: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill monitor-experiment -g -a cursor -y`
Gemini CLI: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill monitor-experiment -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add wanshuiyin/Auto-claude-code-research-in-sleep`

## Description

Monitor running experiments, check progress, collect results. Use when user says "check results", "is it done", "monitor", or wants experiment output.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Monitor Experiment Results

> ⏱ **External cadence is appropriate here.** This skill waits on an external
> fact (job completion / progress), so it is a natural `/loop` / `CronCreate`
> surface: the wake reads status and self-judges only **machine-checkable**
> completion (exit code, file exists, epoch logged), never quality. This is
> the additive external-wait shape in
> [`shared-references/external-cadence.md`](../shared-references/external-cadence.md).
> If a scheduled wait here ends in a verdict step (e.g. then audit results),
> run that verdict **once** after the wait clears, not re-entered per tick.

Monitor: $ARGUMENTS

## Workflow

### Step 1: Check What's Running

**SSH server:**
```bash
ssh <server> "screen -ls"
```

**Vast.ai instance** (read `ssh_host`, `ssh_port` from `vast-instances.json`):
```bash
ssh -p <PORT> root@<HOST> "screen -ls"
```

Also check vast.ai instance status:
```bash
vastai show instances
```

**Modal** (when `gpu: modal` in CLAUDE.md):
```bash
modal app list         # List running/recent apps
modal app logs <app>   # Stream logs from a running app
```
Modal apps auto-terminate when done, if it's not in the list, it already finished. Check results via `modal volume ls <volume>` or local output.

### Step 2: Collect Output from Each Screen
For each screen session, capture the last N lines:
```bash
ssh <server> "screen -S <name> -X hardcopy /tmp/screen_<name>.txt && tail -50 /tmp/screen_<name>.txt"
```

If hardcopy fails, check for log files or tee output.

### Step 3: Check for JSON Result Files
```bash
ssh <server> "ls -lt <results_dir>/*.json 2>/dev/null | head -20"
```

If JSON results exist, fetch and parse them:
```bash
ssh <server> "cat <results_dir>/<latest>.json"
```

### Step 3.5: Pull W&B Metrics (when `wandb: true` in CLAUDE.md)

**Skip this step entirely if `wandb` is not set or is `false` in CLAUDE.md.**

Pull training curves and metrics from Weights & Biases via Python API:

The rest: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep/blob/HEAD/skills/monitor-experiment/SKILL.md