Connecting Nemoclaw to self-hosted vLLM model on same host

[jhthompson12]

Description

using https url

I've successfully nemoclaw onboard my self-hosted vllm instance of gpt-oss-120b using the 3) Other OpenAI-compatible endpoint option with:

OpenAI-compatible base URL (e.g., https://openrouter.ai/api/v1): https://gpt-oss-120b.ai-lab.my-company.com/v1
Other OpenAI-compatible endpoint model []: openai/gpt-oss-120b

But when I try to interact with this model through openclaw tui i get:

run error: LLM request timed out.
connected | error
agent main | session main (openclaw-tui) | inference/openai/gpt-oss-120b | tokens ?/131k

and within the logs (nemoclaw ai-lab-test logs --follow) I see this each time i attempt inference:

[1774449585.801] [sandbox] [WARN ] [openshell_sandbox::proxy] inference endpoint detected but upstream service failed error=upstream unavailable: failed to connect to https://gpt-oss-120b.ai-lab.my-company.com/v1/responses: error sending request for url (https://gpt-oss-120b.ai-lab.my-company.com/v1/responses)

I know the /v1/responses endpoint is working because I've tested it with cURL from my local machine. but If i send the same test cURL request from inside the ai-lab-test sandbox I see this in the nemoclaw logs:

[1774450226.358] [sandbox] [WARN ] [openshell_sandbox::proxy] TLS relay error error=invalid peer certificate: UnknownIssuer host=gpt-oss-120b.ai-lab.my-compant.com port=443

Is the self-signed cert the issue causing all the problems here? How can I add this custom cert to the nemoclaw instance?

using localhost url

I've also tried adding a new provider for the same service, but using the localhost url to the vLLM container which is running on the same host (``)

openshell provider create --name ai-lab-gpt-oss-120b-localhost --type openai --credential "OPENAI_API_KEY=dummy" --config "OPENAI_BASE_URL=http://localhost:8001/v1"

openshell inference set --provider ai-lab-gpt-oss-120b-localhost --model openai/gpt-oss-120b

Error:   × failed to verify inference endpoint for provider 'ai-lab-gpt-oss-120b-localhost' and model 'openai/gpt-oss-120b' at 'http://localhost:8001/v1': failed to
  │ connect to http://localhost:8001/v1/chat/completions: error sending request for url (http://localhost:8001 v1/chat/completions). Next steps: check that the
  │ service is running, confirm the base URL and protocol, and verify credentials, or retry with '--no-verify' if you want to skip verification

but i know this endpoint is also working.

Any help on verifying this is possible or getting this configured would be greatly appreciated!

Reproduction Steps

nemoclaw onboard my model provider successfully. test the model by saying "hello" in the openclaw tui. No inference and receive run error: LLM request timed out. message

Environment

• OS: Ubuntu 22.04.5 LTS
• Node: v22.22.1
• docker: 29.0.4
• nemoclaw: v0.1.0

Debug Output

Logs

Checklist

• I confirmed this bug is reproducible
• I searched existing issues and this is not a duplicate

[jhthompson12]

I was able to uninstall Nemoclaw, set NEMOCLAW_EXPERIMENTAL=1, and reinstall and select the local vLLM option during the provider creation. this is working! BUT we have many vLLM instances running on this host, right now we have Gemma3-27b up on port 8000, but I was hoping to connect to gpt-oss-120b on 8001, or other models on other arbitrary ports.

It looks like with this approach I'm stuck with port 8000? Is there a way to modify the port for the local vLLM option? Otherwise, it would be great to get the approach mentioned above working where I could provide an arbitrary URL for a vLLM instance and provide self signed certificate details as needed.

[0x1stvan]

In the early versions, when I tested OpenClaw with vLLM, I had to use the URL

http://host.openshell.internal:8000/v1

instead of the

http://localhost:8000/v1

[jhthompson12]

thank you, @0x1stvan! I had tried http://host.docker.internal:8001/v1and wasnt aware of the url you shared. So i gave it a shot.

TL;DR: it does work, but it took a roundabout way to get it configured.. perhaps this is due to a bug of some kind?

Here's the steps i tried that failed

From nemoclaw onboard if I use http://host.openshell.internal:8001/v1 it runs into some issue validating the endpoint and will not let me proceed:

  Detected local inference option: vLLM
  Select one explicitly to use it. Press Enter to keep NVIDIA Endpoints.


  Inference options:
    1) NVIDIA Endpoints (recommended)
    2) OpenAI
    3) Other OpenAI-compatible endpoint
    4) Anthropic
    5) Other Anthropic-compatible endpoint
    6) Google Gemini

  Choose [1]: 3
  OpenAI-compatible base URL (e.g., https://openrouter.ai/api/v1): http://host.openshell.internal:8001/v1
  Other OpenAI-compatible endpoint model []: openai/gpt-oss-120b
  Other OpenAI-compatible endpoint endpoint validation failed.
  Responses API: HTTP 6 with no response body | Chat Completions API: HTTP 6 with no response body
  Please choose a provider/model again.

  Detected local inference option: vLLM
  Select one explicitly to use it. Press Enter to keep NVIDIA Endpoints.

If instead, I use the reverse proxy URL for the same service (https://gpt-oss-120b.ai-lab.my-company.com/v1) the validation passes and I can move onto the finishing setup steps:

  Inference options:
    1) NVIDIA Endpoints (recommended)
    2) OpenAI
    3) Other OpenAI-compatible endpoint
    4) Anthropic
    5) Other Anthropic-compatible endpoint
    6) Google Gemini

  Choose [1]: 3
  OpenAI-compatible base URL (e.g., https://openrouter.ai/api/v1): https://gpt-oss-120b.ai-lab.my-company.com/v1
  Other OpenAI-compatible endpoint model []: openai/gpt-oss-120b
  Responses API available — OpenClaw will use openai-responses.
  Using Other OpenAI-compatible endpoint with model: openai/gpt-oss-120b

  [3/7] Starting OpenShell gateway
  ──────────────────────────────────────────────────
• Destroying gateway nemoclaw...
✓ Gateway nemoclaw destroyed.
  Using pinned OpenShell gateway image: ghcr.io/nvidia/openshell/cluster:0.0.16
✓ Checking Docker
✓ Downloading gateway
✓ Initializing environment
✓ Starting gateway                                                                                                                                                                                                      
✓ Gateway ready

  Name: nemoclaw
  Endpoint: https://127.0.0.1:8080

✓ Active gateway set to 'nemoclaw'
  ✓ Gateway is healthy
✓ Active gateway set to 'nemoclaw'

  [4/7] Setting up inference provider
  ──────────────────────────────────────────────────
✓ Active gateway set to 'nemoclaw'
✓ Created provider compatible-endpoint
Gateway inference configured:

  Route: inference.local
  Provider: compatible-endpoint
  Model: openai/gpt-oss-120b
  Version: 1
  ✓ Inference route set: compatible-endpoint / openai/gpt-oss-120b

But subsequent interactions with the model through the openclaw tui fail as highlighted in my initial post above:

failed to connect to https://gpt-oss-120b.ai-lab.my-company.com/v1/responses: error sending request for url (https://gpt-oss-120b.ai-lab.my-company.com/v1/responses)

Now try to swap out the provider URLs

openshell provider create --name ai-lab-gpt-oss-120b-openshell-url --type openai --credential "OPENAI_API_KEY=dummy" --config "OPENAI_BASE_URL=http://host.openshell.internal:8001/v1"
openshell inference set --provider ai-lab-gpt-oss-120b-openshell-url --model openai/gpt-oss-120b
openshell provider delete compatible-endpoint

This time the http://host.openshell.internal:8001/v1provider is validated with no issues:

Gateway inference configured:

  Route: inference.local
  Provider: ai-lab-gpt-oss-120b-openshell-url
  Model: openai/gpt-oss-120b
  Version: 2
  Validated Endpoints:
    - http://host.openshell.internal:8001/v1/chat/completions (openai_chat_completions)

then if I go back into the sandbox and enter openclaw tui, the inference is working! here's the line from from nemoclaw my-assistant logs --follow as proof:

[1774535924.090] [sandbox] [INFO ] [openshell_router] routing proxy inference request (streaming) endpoint=http://host.openshell.internal:8001/v1 method=POST path=/v1/responses protocols=openai_chat_completions,openai_completions,openai_responses,model_discovery

So maybe this is a issue/bug with the installer/onboarding?

[0x1stvan]

You can try to use the --no-verify flag when you set the inference:
https://docs.nvidia.com/openshell/latest/inference/configure.html#verify-the-endpoint-from-a-sandbox

openshell inference set --provider <provider-name> --model <model-name> --no-verify

[hodgesz]

Same issue with Ollama on macOS (Apple Silicon / Docker Desktop)

Adding another data point — hit the same core problem trying to use Ollama as a local inference provider on macOS with Docker Desktop.

Environment

• OS: macOS 15 (Apple Silicon M4)
• Docker: Docker Desktop for Mac (linux/arm64)
• Node: v22.16.0
• NemoClaw: built from source (main branch, commit eb4ba8c)
• OpenShell: 0.0.16
• Ollama: running on host, port 11434

What NemoClaw does during onboard

bin/lib/onboard.js correctly creates the provider with:

openshell provider create --name ollama-local --type openai \
  --credential OPENAI_API_KEY \
  --config "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1"

And sets inference with:

openshell inference set --no-verify --provider ollama-local --model <model>

What actually happens

The gateway ignores OPENAI_BASE_URL and routes to api.openai.com:

[openshell_router] routing proxy inference request
  endpoint=https://api.openai.com/v1
  method=POST path=/v1/chat/completions

This results in a 401 because the dummy API key (ollama) is sent to the real OpenAI API instead of to Ollama on the host.

Workaround that matches @jhthompson12's finding

Same as what jhthompson12 discovered above — if you manually create the provider with host.openshell.internal after onboarding, the gateway does respect the URL:

openshell provider create --name ollama-manual \
  --type openai \
  --credential "OPENAI_API_KEY=ollama" \
  --config "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1"

openshell inference set --provider ollama-manual --model <model> --no-verify

But this shouldn't be necessary — nemoclaw onboard already passes these exact same arguments. Something in the provider creation or inference routing pipeline is dropping the base URL config under certain conditions.

Additional context

• PR fix: restore routed inference and connect UX #1037 (now on main) fixed NO_PROXY not being exported, which was one layer of this problem, but doesn't resolve the base URL being ignored.
• The README.md (line 223) already acknowledges: "local host-routed inference on macOS still depends on OpenShell host-routing support"
• The ollama_local network policy in openclaw-sandbox.yaml correctly allows host.openshell.internal:11434, so the policy layer isn't the blocker — it's the gateway routing layer.
• This likely affects anyone trying local inference (Ollama, vLLM, etc.) through the standard onboarding flow on any platform.

Related issues

• fix: inference.local does not resolve inside sandbox on macOS (Docker Desktop) #1058 — Ollama onboard fails on macOS
• inference.local routing not created for sandboxes after openshell inference set #326 — host.openshell.internal DNS resolution
• Local Ollama inference blocked by OpenShell egress proxy — RFC1918 addresses return 403 regardless of policy #693 — inference.local proxy bypass on macOS
• macOS/Apple Silicon Support Tracking — Known Gaps & Fixes #260 — NO_PROXY misconfiguration (partially fixed by fix: restore routed inference and connect UX #1037)

[dineshtripathi30]

thank you, @0x1stvan! I had tried http://host.docker.internal:8001/v1and wasnt aware of the url you shared. So i gave it a shot.

TL;DR: it does work, but it took a roundabout way to get it configured.. perhaps this is due to a bug of some kind?

Here's the steps i tried that failed

From nemoclaw onboard if I use http://host.openshell.internal:8001/v1 it runs into some issue validating the endpoint and will not let me proceed:

  Detected local inference option: vLLM
  Select one explicitly to use it. Press Enter to keep NVIDIA Endpoints.


  Inference options:
    1) NVIDIA Endpoints (recommended)
    2) OpenAI
    3) Other OpenAI-compatible endpoint
    4) Anthropic
    5) Other Anthropic-compatible endpoint
    6) Google Gemini

  Choose [1]: 3
  OpenAI-compatible base URL (e.g., https://openrouter.ai/api/v1): http://host.openshell.internal:8001/v1
  Other OpenAI-compatible endpoint model []: openai/gpt-oss-120b
  Other OpenAI-compatible endpoint endpoint validation failed.
  Responses API: HTTP 6 with no response body | Chat Completions API: HTTP 6 with no response body
  Please choose a provider/model again.

  Detected local inference option: vLLM
  Select one explicitly to use it. Press Enter to keep NVIDIA Endpoints.

If instead, I use the reverse proxy URL for the same service (https://gpt-oss-120b.ai-lab.my-company.com/v1) the validation passes and I can move onto the finishing setup steps:

  Inference options:
    1) NVIDIA Endpoints (recommended)
    2) OpenAI
    3) Other OpenAI-compatible endpoint
    4) Anthropic
    5) Other Anthropic-compatible endpoint
    6) Google Gemini

  Choose [1]: 3
  OpenAI-compatible base URL (e.g., https://openrouter.ai/api/v1): https://gpt-oss-120b.ai-lab.my-company.com/v1
  Other OpenAI-compatible endpoint model []: openai/gpt-oss-120b
  Responses API available — OpenClaw will use openai-responses.
  Using Other OpenAI-compatible endpoint with model: openai/gpt-oss-120b

  [3/7] Starting OpenShell gateway
  ──────────────────────────────────────────────────
• Destroying gateway nemoclaw...
✓ Gateway nemoclaw destroyed.
  Using pinned OpenShell gateway image: ghcr.io/nvidia/openshell/cluster:0.0.16
✓ Checking Docker
✓ Downloading gateway
✓ Initializing environment
✓ Starting gateway                                                                                                                                                                                                      
✓ Gateway ready

  Name: nemoclaw
  Endpoint: https://127.0.0.1:8080

✓ Active gateway set to 'nemoclaw'
  ✓ Gateway is healthy
✓ Active gateway set to 'nemoclaw'

  [4/7] Setting up inference provider
  ──────────────────────────────────────────────────
✓ Active gateway set to 'nemoclaw'
✓ Created provider compatible-endpoint
Gateway inference configured:

  Route: inference.local
  Provider: compatible-endpoint
  Model: openai/gpt-oss-120b
  Version: 1
  ✓ Inference route set: compatible-endpoint / openai/gpt-oss-120b

But subsequent interactions with the model through the openclaw tui fail as highlighted in my initial post above:

failed to connect to https://gpt-oss-120b.ai-lab.my-company.com/v1/responses: error sending request for url (https://gpt-oss-120b.ai-lab.my-company.com/v1/responses)

Now try to swap out the provider URLs

openshell provider create --name ai-lab-gpt-oss-120b-openshell-url --type openai --credential "OPENAI_API_KEY=dummy" --config "OPENAI_BASE_URL=http://host.openshell.internal:8001/v1"
openshell inference set --provider ai-lab-gpt-oss-120b-openshell-url --model openai/gpt-oss-120b
openshell provider delete compatible-endpoint

This time the http://host.openshell.internal:8001/v1provider is validated with no issues:

Gateway inference configured:

  Route: inference.local
  Provider: ai-lab-gpt-oss-120b-openshell-url
  Model: openai/gpt-oss-120b
  Version: 2
  Validated Endpoints:
    - http://host.openshell.internal:8001/v1/chat/completions (openai_chat_completions)

then if I go back into the sandbox and enter openclaw tui, the inference is working! here's the line from from nemoclaw my-assistant logs --follow as proof:

[1774535924.090] [sandbox] [INFO ] [openshell_router] routing proxy inference request (streaming) endpoint=http://host.openshell.internal:8001/v1 method=POST path=/v1/responses protocols=openai_chat_completions,openai_completions,openai_responses,model_discovery

So maybe this is a issue/bug with the installer/onboarding?

I had exactly same issue and similar fix working.. also sounds to me like installer/onboarding issue

[hodgesz]

Update: The delete-and-recreate workaround works for Ollama on macOS too.

Steps that fixed it:

# Delete the provider created during nemoclaw onboard
openshell provider delete ollama-local

# Recreate it manually with the exact same args
openshell provider create --name ollama-local --type openai \
  --credential "OPENAI_API_KEY=ollama" \
  --config "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1"

# Set inference route
openshell inference set --provider ollama-local --model nemotron-mini:latest --no-verify

Logs now confirm correct routing:

[openshell_router] routing proxy inference request (streaming)
  endpoint=http://host.openshell.internal:11434/v1
  method=POST path=/v1/chat/completions

This confirms the bug is in the onboarding flow, not in OpenShell's gateway routing itself. The gateway can route to host.openshell.internal — it just doesn't when the provider is created through nemoclaw onboard. Something in that code path is either not persisting the OPENAI_BASE_URL config or the gateway is picking up a stale provider definition.