Skills

Iterative Debugging Behavior

Block-by-block fix loop on BEHAVIOR-1K (R1 Pro, OmniGibson on Isaac Sim).

Install

npx skills add NVlabs/ASPIRE --skill iterative-debugging-behavior -g -a claude-code -y
Repository
NVlabs/ASPIRE925
Category
Engineering
Updated
20d ago
License
Apache-2.0
Context cost
Mediumabout 1,817 tokens
Runs scripts
Yes, review before use

What it does

Block-by-block fix loop on BEHAVIOR-1K (R1 Pro, OmniGibson on Isaac Sim). The policy is grown one code block at a time against what the robot actually did, and what carries across seeds is a skill library, not a program. Use for the two R1 Pro pickup tasks; Stage 1 learns on seeds 26-35, Stage 2 evaluates on 1-25 with a frozen library.

Block-by-Block Fix Loop on BEHAVIOR-1K

The parent skill (../SKILL.md) writes one whole program per task, freezes it, and replays that frozen program across the held-out seeds. That protocol does not apply here. BEHAVIOR tasks are long-horizon: the robot starts in another room, cannot see the target, and must search, drive, look down and grasp. A program authored up front is guessing at every stage past the first.

So on this benchmark the policy is grown, not authored:

  1. Write one # Code block N section.
  2. Run the whole policy.py on the seed, from a fresh reset.
  3. Read what actually happened: where the base ended up, what was localized, what the hand holds.
  4. Write the next block against that observed state, and run the whole program again.
  5. Repeat until the seed succeeds or the budget is spent.

The simulator resets on every attempt. Only policy.py persists and accumulates blocks.

What carries across seeds is the skill library, not any policy. Each seed's policy.py is disposable. Stage 2 measures whether the frozen library is enough for a fresh agent to solve an unseen seed, not whether one program generalizes.

The library is shared by both tasks. It lives at $RUN_ROOT/behavior/skill-library/, above the suite directories, and a task's Stage 1 extends what the previous task left. This is a deliberate departure from ASPIRE, which keeps one library per task. The reason is that most of what a campaign learns here is about the harness, the robot and the simulator rather than about one object, and re-discovering it per task wastes a bootstrap seed each time. It also asks a more interesting question: does a library built on one task help on a different one.

It leaks nothing, because the partition is per task. Knowledge measured on the radio's seeds says nothing about a soda-can instance, so the can's evaluation seeds stay unseen.

Read the whole SKILL.md on GitHub

More in Engineering