Fix: Claude Sonnet 4.6 rejects requests with both temperature and top_p

[juanmichelini]

Problem

Claude Sonnet 4.6 (and other Anthropic models) reject API requests when both temperature AND top_p are specified together.

Evidence from Production Failures

Multiple evaluation jobs failing with 100% failure rate:

• eval-22184270397-claude-son (failed after 7h32m, swebench)
• eval-22184282669-claude-son (failed after 7h32m, commit0)
• eval-22112470055-claude-son (error state, 21h47m, swtbench)

Root Cause

1. SDK default: top_p=1.0 (set in openhands/sdk/llm/llm.py)
2. Benchmarks override: temperature=0.1 (set in evaluation configs)
3. Result: Both parameters sent to Anthropic API → request rejected

From Anthropic's API docs:

You cannot specify both temperature and top_p in the same request.

Current Code

# openhands-sdk/openhands/sdk/llm/llm.py
class LLM(BaseModel):
    top_p: float | None = Field(
        default=1.0,  # ← Always sends this
        ge=0,
        le=1,
        description="Top-p (nucleus) sampling parameter..."
    )

Proposed Minimal Fix

Option A: Don't send top_p for Claude models when temperature is set

Add logic to skip top_p for Anthropic models when temperature is explicitly provided:

# In model_features.py or similar
def should_send_top_p(model: str, temperature: float | None) -> bool:
    """Return False if model doesn't support both temperature and top_p together."""
    # Anthropic models (Claude) don't accept both parameters
    if 'claude' in model.lower() or 'anthropic' in model.lower():
        return temperature is None
    return True

Option B: Set top_p default to None for Claude models

def get_default_top_p(model: str) -> float | None:
    """Return None for Claude models to avoid conflicts with temperature."""
    if 'claude' in model.lower() or 'anthropic' in model.lower():
        return None
    return 1.0  # Keep existing default for other models

Scope

This issue is ONLY about fixing Claude Sonnet 4.6 compatibility.

• ✅ Fix Claude models specifically
• ✅ Minimal code change
• ✅ No breaking changes for other models
• ❌ NOT addressing broader default value changes (see PR Set default temperature to None instead of 0.0 #1989 for that)

Related

• Broader fix: PR Set default temperature to None instead of 0.0 #1989 (changes defaults for all models, has breaking change concerns)
• Related issue: RFC: Always set temperature to default? #1913

Impact

Currently blocking production evaluations:

• 100% failure rate for Claude Sonnet 4.6 evaluations

[juanmichelini]

@OpenHands explain why are we getting this issue with claude-sonnet-4-6 and not claude-sonnet-4-5 or claude-opus-4-6

[openhands-ai]

I'm on it! juanmichelini can track my progress at all-hands.dev

[openhands-ai]

Summary: Why claude-sonnet-4-6 Fails While Other Claude Models Work

I investigated the issue and found the root cause:

The Problem

claude-sonnet-4-6 is missing from the model feature registry, causing it to send both temperature AND top_p parameters to Anthropic's API, which rejects requests with both parameters set.

Why Other Models Work

claude-opus-4-6:

• Listed in REASONING_EFFORT_MODELS (model_features.py line 73)
• The SDK automatically strips both temperature and top_p for reasoning models
• ✅ Only valid parameters sent to Anthropic

claude-sonnet-4-5:

• Listed in EXTENDED_THINKING_MODELS (model_features.py line 82)
• The SDK automatically strips both temperature and top_p for extended thinking models
• ✅ Only valid parameters sent to Anthropic

claude-sonnet-4-6:

• ❌ NOT listed in either category
• Receives both temperature=0.0 (from eval config) AND top_p=1.0 (SDK default)
• Anthropic API rejects the request → 100% evaluation failure rate

The Code Flow

1. llm.py line 187: Sets default top_p=1.0
2. Evaluation config sets temperature=0.0 for claude-sonnet-4-6
3. chat_options.py lines 36-44 & 52-72: Only removes both params for recognized feature flags
4. Since claude-sonnet-4-6 has no feature flag, both params are sent
5. Anthropic API rejects: "You cannot specify both temperature and top_p"

Solution Needed

Add "claude-sonnet-4-6" to either:

• EXTENDED_THINKING_MODELS list, or
• REASONING_EFFORT_MODELS list, or
• Implement generic Anthropic handling to never send both parameters together

This explains why only claude-sonnet-4-6 evaluations are failing while the other Claude models work fine! 🎯

[juanmichelini]

@OpenHands okay let's fix this by adding claude-sonnet-4-6 to EXTENDED_THINKING_MODELS same as we did with claude-sonnet-4-5

[openhands-ai]

I'm on it! juanmichelini can track my progress at all-hands.dev