HomeSafe-Bench, Indoor Safety Hazard Detection

VLM benchmark evaluating indoor home safety hazard detection from static camera frames. Inspired by HomeSafeBench (arXiv 2509.23690), adapted for fixed indoor security cameras with AI-generated fixture images.

40 tests across 5 hazard categories:

Category	Tests	Example Hazards
🔥 Fire / Smoke	8	Stove smoke, candle near curtain, iron left face-down
⚡ Electrical	8	Overloaded power strip, exposed wire, frayed cord
🚶 Trip / Fall	8	Toys on staircase, wet floor, loose rug edge
👶 Child Safety	8	Open cabinet with chemicals, uncovered outlet, blind cord
📦 Falling Objects	8	Heavy items on high shelf, leaning bookshelf, loose ceiling fan

Setup

Requires npm install. This skill has a package.json with dependencies (e.g. openai SDK). The deployment agent should check for package.json and run npm install in the skill directory before launching.

Entry script: scripts/run-benchmark.cjs

Verification

node scripts/run-benchmark.cjs --help

Quick Start

As an Aegis Skill (automatic)

When spawned by Aegis, all configuration is injected via environment variables. The benchmark discovers your VLM server automatically, generates an HTML report, and opens it when complete.

Standalone

# Run all 40 tests
node scripts/run-benchmark.cjs --vlm http://localhost:5405

# Quick mode (2 tests per category = 10 total)
node scripts/run-benchmark.cjs --vlm http://localhost:5405 --mode quick

# Skip report auto-open
node scripts/run-benchmark.cjs --vlm http://localhost:5405 --no-open

Configuration

Environment Variables (set by Aegis)