OpenRouter accepts invalid API keys as enabled

[ElioNeto]

Original issue tinyhumansai#2366 by @Al629176 on 2026-05-20T19:11:19Z

---

Summary

OpenRouter can be enabled with a random invalid API key, so users think the provider is connected even though it was never validated.

Problem

In the custom/cloud provider setup flow, entering a random API key for OpenRouter is accepted and the OpenRouter provider toggle becomes enabled. The app does not appear to validate the key or endpoint before marking the provider usable.

Expected behavior: OpenRouter should only be enabled after the key and endpoint are validated, or it should clearly show an unverified/pending state until validation succeeds.

Actual behavior: a random key is accepted and OpenRouter appears enabled.

Impact: users can configure a broken provider and only discover the failure later in chat or routing, where the cause is much harder to understand.

Steps to reproduce:

1. Open Settings → AI → LLM.
2. Add a cloud provider.
3. Select OpenRouter.
4. Enter a random invalid API key.
5. Save the provider.
6. Observe that OpenRouter is enabled despite the invalid key.

Version / platform: macOS desktop app, screenshot captured May 21, 2026. Exact app version unknown.

Solution (optional)

Validate provider credentials before enabling the provider. For OpenRouter, use the correct OpenRouter endpoint and a lightweight validation call. If offline validation is not possible, mark the provider as “saved but unverified” rather than enabled.

Acceptance criteria

• Invalid key rejected — A random OpenRouter key does not enable OpenRouter as a usable provider.
• Correct provider endpoint — OpenRouter validation uses the OpenRouter endpoint, not an OpenAI endpoint.
• Clear state — Saved but unverified credentials are shown as pending/unverified, not connected/enabled.
• Routing protected — Routing does not select an unverified provider for chat or background workloads.
• Regression safety — Tests cover invalid key, valid mocked key, and endpoint mismatch behavior.
• Diff coverage ≥ 80% — the fix PR meets the changed-lines coverage gate (Vitest + cargo-llvm-cov, enforced by .github/workflows/coverage.yml).

Related

• Screenshot: Screenshot 2026-05-21 at 12.34.59 AM.png

[ElioNeto]

Comment by @sunilkumarvalmiki on 2026-05-20T19:53:18Z

I reproduced this on the current code path and can take it.

What I found:

• The AI settings flow stores the OpenRouter key, flushes the provider config, then verifies by calling openhuman.inference_list_models.
• That controller calls GET {endpoint}/models through src/openhuman/inference/provider/ops.rs.
• OpenRouter's model catalog is not a credential check. It returns 200 for both no auth and an obviously invalid bearer:

curl.exe -sS -o NUL -w "%{http_code}" https://openrouter.ai/api/v1/models
# 200

curl.exe -sS -o NUL -w "%{http_code}" -H "Authorization: Bearer definitely-invalid-openrouter-key" https://openrouter.ai/api/v1/models
# 200

• The credential check needs to use OpenRouter's current-key endpoint instead. With the same invalid bearer, GET https://openrouter.ai/api/v1/key returns 401:

curl.exe -sS -o NUL -w "%{http_code}" -H "Authorization: Bearer definitely-invalid-openrouter-key" https://openrouter.ai/api/v1/key
# 401

I am going to work on a fix that keeps /models as the model-catalog compatibility probe, adds an OpenRouter-specific credential validation step before the provider is marked enabled, rolls back the saved key/config when validation fails, and adds regression coverage for the /models-200 + /key-401 case.

[ElioNeto]

Comment by @sunilkumarvalmiki on 2026-05-20T20:43:04Z

Fix PR is open: tinyhumansai#2372

What changed:

• OpenRouter providers now validate the configured key with GET {endpoint}/key before the existing /models catalog probe runs.
• The configured key is normalized once and reused for both /key and /models requests.
• The /models probe is still used after credential validation so valid keys keep the same model-list behavior.
• Non-OpenRouter OpenAI-compatible providers keep the existing /models validation path.
• Validation errors are sanitized/truncated and the stored key is not logged.

After-fix evidence:

cargo fmt --manifest-path Cargo.toml --all --check
# passed

cargo test --manifest-path Cargo.toml openrouter_ --lib -- --nocapture
# 5 passed; 0 failed

pnpm --filter openhuman-app test -- src/components/settings/panels/__tests__/AIPanel.test.tsx src/services/api/__tests__/aiSettingsApi.test.ts
# 2 files passed; 60 tests passed

pnpm typecheck
# passed

cargo check --manifest-path Cargo.toml --lib
# passed with existing warnings

The regression tests cover the original failure shape: /key returns 401, /models would return a valid catalog, and the controller now fails before calling /models. They also cover the valid-key path and the key-normalization path.