Fix: Security analyzer ignores LLM security_risk when no analyzer is configured

[juanmichelini]

Summary

Fixes #1957

Previously, when llm_security_analyzer=False (or when no security analyzer was configured via conversation.set_security_analyzer()), the agent would still evaluate and use the security_risk value provided by the LLM. This caused actions to be incorrectly flagged with security risks even when security analysis was explicitly disabled.

Root Cause

The _extract_security_risk() method in agent.py was extracting and using the LLM-provided security_risk value regardless of whether a security analyzer was configured. The code only checked if the analyzer was an instance of LLMSecurityAnalyzer but didn't handle the case when security_analyzer is None.

Solution

Modified _extract_security_risk() to return SecurityRisk.UNKNOWN when security_analyzer is None, effectively ignoring any security_risk value provided by the LLM when no analyzer is configured.

Why add_security_risk_prediction=True is NOT changed

The add_security_risk_prediction=True parameter in openhands/sdk/agent/utils.py is intentionally kept unchanged. This parameter ensures the security_risk field is always included in tool schemas for consistency, even when the security analyzer is disabled. The schema consistency helps:

1. LLMs have a consistent tool signature to work with
2. Weaker models can still omit the field without breaking validation
3. Runtime behavior is controlled by _extract_security_risk(), not the schema

The fix ensures that the field is properly ignored at runtime when no analyzer is set, which is the correct behavior as documented in the code comments.

Changes

1. openhands-sdk/openhands/sdk/agent/agent.py: Added early return in _extract_security_risk() to return UNKNOWN when security_analyzer is None
2. tests/sdk/agent/test_extract_security_risk.py: Updated test expectations to verify that security_risk is ignored (returns UNKNOWN) when no analyzer is set
3. tests/sdk/agent/test_security_policy_integration.py: Enhanced existing test to verify the fix for the reported issue

Checklist

• If the PR is changing/adding functionality, are there tests to reflect this?
  • Yes, updated existing tests and enhanced integration test
• If there is an example, have you run the example to make sure that it works?
  • N/A - this is a bug fix, not a new example
• If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
  • Yes, all tests pass
• If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
  • N/A - this is a bug fix that restores expected behavior, no doc changes needed
• Is the github CI passing?
  • Pre-commit checks passed locally, awaiting CI

@juanmichelini can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.12-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:457a5b6-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-457a5b6-python \
  ghcr.io/openhands/agent-server:457a5b6-python

All tags pushed for this build

ghcr.io/openhands/agent-server:457a5b6-golang-amd64
ghcr.io/openhands/agent-server:457a5b6-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:457a5b6-golang-arm64
ghcr.io/openhands/agent-server:457a5b6-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:457a5b6-java-amd64
ghcr.io/openhands/agent-server:457a5b6-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:457a5b6-java-arm64
ghcr.io/openhands/agent-server:457a5b6-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:457a5b6-python-amd64
ghcr.io/openhands/agent-server:457a5b6-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:457a5b6-python-arm64
ghcr.io/openhands/agent-server:457a5b6-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:457a5b6-golang
ghcr.io/openhands/agent-server:457a5b6-java
ghcr.io/openhands/agent-server:457a5b6-python

About Multi-Architecture Support

• Each variant tag (e.g., 457a5b6-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., 457a5b6-python-amd64) are also available if needed

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/agent
agent.py	233	36	84%	94, 98, 238–240, 242, 272–273, 280–281, 313, 366–367, 369, 409, 554–555, 560, 572–573, 578–579, 598–599, 601, 629–630, 637–638, 642, 650–651, 688, 694, 706, 713
TOTAL	18294	5563	69%

[all-hands-bot]

🟢 Good taste - Clean, pragmatic bug fix. The early return in _extract_security_risk() properly separates "schema includes field" from "runtime uses field value". Tests reproduce the actual bug from #1957 and verify the fix. LGTM!

[juanmichelini]

@OpenHands @XZ-X has tested it but still sees “security risk” as “low”, not “unknown” as expected from this PR. Please double check the tests are testing correctly, and rerun them. Also do a manual integration test to first reproduce the issue, then make sure the fix addresses the issue. If not reconsider it.

[openhands-ai]

I'm on it! juanmichelini can track my progress at all-hands.dev

[enyst]

Why did the LLM return any security risk, if no analyzer was configured?