# Debug Inference

Debug inference clients that use an attached provider and its native endpoint...

- Repository: https://github.com/NVIDIA/OpenShell (20903 stars)
- Category: Testing
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add NVIDIA/OpenShell --skill debug-inference -g -a claude-code -y`
Codex: `npx skills add NVIDIA/OpenShell --skill debug-inference -g -a codex -y`
Cursor: `npx skills add NVIDIA/OpenShell --skill debug-inference -g -a cursor -y`
Gemini CLI: `npx skills add NVIDIA/OpenShell --skill debug-inference -g -a gemini-cli -y`

## Description

Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM. Use for provider attachment, endpoint policy, credential substitution, topology, and migration from the removed managed inference endpoint. Trigger keywords - debug inference, managed inference endpoint, local inference, ollama, lm studio, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, credential_endpoint_mismatch, host.openshell.internal.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Debug Inference

Diagnose inference as ordinary provider-authorized network traffic. OpenShell no
longer supplies a managed inference route, rewrites request shapes, or selects a
model. The application calls the provider's native endpoint and owns its base
URL, model, request format, and timeout.

Use installed `openshell --help` output as the authority for command syntax.
Refer to the published [provider management guide](https://docs.nvidia.com/openshell/latest/how-it-works/providers/overview)
and [provider profile guide](https://docs.nvidia.com/openshell/latest/how-it-works/providers/profiles)
for current behavior.

## Diagnostic Workflow

### 1. Confirm Gateway and Sandbox Context

```bash
openshell status
openshell gateway info
openshell sandbox get <sandbox>
```

For a host-local model server, `host.openshell.internal` identifies the machine
running the gateway. It does not identify the operator's laptop when the gateway
is remote. A server listening only on `127.0.0.1` may also be unreachable from a
container; bind it to an address reachable from the gateway runtime.

### 2. Inspect the Provider and Its Profile

```bash
openshell provider get <provider>
openshell profile export <profile-id> -o yaml
```

Check that the profile:

- Names the exact endpoint host, port, and protocol the client calls.
- Allows the client binary.
- Declares the credential key and intended authentication style.
- Uses narrow HTTP rules when the provider should expose only part of an API.

For a custom or self-hosted OpenAI-compatible endpoint, import an
endpoint-bearing profile. A base URL stored only in provider configuration does
not authorize a new endpoint.

```bash
openshell profile lint -f ./provider-profile.yaml
openshell profile import -f ./provider-profile.yaml
openshell provider create --name <provider> --type <profile-id>
```

The rest: https://github.com/NVIDIA/OpenShell/blob/HEAD/skills/debug-inference/SKILL.md