# Perception Localize

Locate objects with a running PerceptionLocalize stack over MCP (`state`, `localize`) -...

- Repository: https://github.com/dimensionalOS/dimos (4702 stars)
- Category: AI and agents
- License: see repository
- Last updated: 2026-11-01
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add dimensionalOS/dimos --skill perception-localize -g -a claude-code -y`
Codex: `npx skills add dimensionalOS/dimos --skill perception-localize -g -a codex -y`
Cursor: `npx skills add dimensionalOS/dimos --skill perception-localize -g -a cursor -y`
Gemini CLI: `npx skills add dimensionalOS/dimos --skill perception-localize -g -a gemini-cli -y`

## Description

Locate objects with a running PerceptionLocalize stack over MCP (`state`, `localize`) - probe the machine, build the time window, watch labels on a schedule, narrow an object's position, change params per call, and tell where a failure comes from. Use whenever a blueprint with PerceptionLocalize and McpServer runs, such as go2-localize or xarm-localize-live.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Localize over MCP

`localize` answers from memory: the colour frames the module has embedded,
lifted to 3D through depth or lidar. It does not look at the current frame.
Every answer is a verified 3D object with a score and a view count, and an
empty answer is an answer.

## Before the first call

    nvidia-smi --query-gpu=name,memory.used,memory.total --format=csv,noheader
    uv run dimos mcp call state

- Probe the GPU first. How many labels a call can carry depends on it, see
  "Labels and GPU".
- `state()` returns `ready: N frames embedded, localize will answer` or
  `not ready: <stage>`. Until it reads ready, `localize` returns its stage
  instead of an answer.
- `Error: no running MCP server (is DimOS running?)` means the stack is down.
  Check `uv run dimos status` and report it. Do not stop or restart the stack.

## Calling localize

    uv run dimos mcp call localize -j '{"objects": "chair,door", "start": -30, "duration": 30}' --timeout 300

- `localize(objects, start=-10, duration=10, params="")` returns once the answer
  is built. Send the next call after it returns.
- `objects` is one label, or a few separated by commas. The labels of one call
  share one detector pass per frame.
- The client waits 30 s. A window with many unread frames takes longer, so pass
  `--timeout 300` or set `MCP_TIMEOUT=300`.

## Building the window

`start` and `duration` are seconds over the embedded frames the module holds.

| arguments | window |
|---|---|
| `start=-10, duration=10` (default) | the last 10 s |
| `start=-60, duration=60` | the last minute |
| `start=-60, duration=20` | from 60 s ago to 40 s ago |
| `start=30, duration=20` | 20 s, from 30 s after the oldest frame held |

The rest: https://github.com/dimensionalOS/dimos/blob/HEAD/dimos/perception/localize/SKILL.md