Debug Inference
Debug inference clients that use an attached provider and its native endpoint...
Install
npx skills add NVIDIA/OpenShell --skill debug-inference -g -a claude-code -y- Repository
- NVIDIA/OpenShell20.9k
- Category
- Testing
- Updated
- 4d ago
- License
- Apache-2.0
- Context cost
- Mediumabout 1,736 tokens
- Runs scripts
- No, instructions only
What it does
Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM. Use for provider attachment, endpoint policy, credential substitution, topology, and migration from the removed managed inference endpoint. Trigger keywords - debug inference, managed inference endpoint, local inference, ollama, lm studio, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, credential_endpoint_mismatch, host.openshell.internal.
Debug Inference
Diagnose inference as ordinary provider-authorized network traffic. OpenShell no longer supplies a managed inference route, rewrites request shapes, or selects a model. The application calls the provider's native endpoint and owns its base URL, model, request format, and timeout.
Use installed openshell --help output as the authority for command syntax.
Refer to the published provider management guide
and provider profile guide
for current behavior.
Diagnostic Workflow
1. Confirm Gateway and Sandbox Context
openshell status
openshell gateway info
openshell sandbox get <sandbox>
For a host-local model server, host.openshell.internal identifies the machine
running the gateway. It does not identify the operator's laptop when the gateway
is remote. A server listening only on 127.0.0.1 may also be unreachable from a
container; bind it to an address reachable from the gateway runtime.
2. Inspect the Provider and Its Profile
openshell provider get <provider>
openshell profile export <profile-id> -o yaml
Check that the profile:
- Names the exact endpoint host, port, and protocol the client calls.
- Allows the client binary.
- Declares the credential key and intended authentication style.
- Uses narrow HTTP rules when the provider should expose only part of an API.
For a custom or self-hosted OpenAI-compatible endpoint, import an endpoint-bearing profile. A base URL stored only in provider configuration does not authorize a new endpoint.
openshell profile lint -f ./provider-profile.yaml
openshell profile import -f ./provider-profile.yaml
openshell provider create --name <provider> --type <profile-id>
