Claude provider: _execute_with_parse_recovery blocks MCP tool calls in agentic loop

[franklixuefei]

Bug Description

When using the Claude provider with MCP servers and agents that have output schemas, MCP tool calls are never executed. The _execute_with_parse_recovery() method intercepts the API response before the agentic loop can process tool calls, enters a futile parse recovery cycle, and fails with "Failed to extract valid JSON after 2 recovery attempts".

Root Cause

In src/conductor/providers/claude.py, _execute_with_parse_recovery() (~line 1132) is called inside the agentic loop when has_output_schema=True. The flow:

1. API call returns tool_use blocks for MCP tools (e.g., filesystem__read_file)
2. _extract_structured_output() checks for emit_output tool_use — returns None (not emit_output)
3. _extract_json_fallback() tries to parse text content — returns None (text is empty or thinking text)
4. Parse recovery fires: sends recovery message asking for JSON
5. Model is confused (it already called a tool), recovery fails
6. After 2 recovery attempts, raises ProviderError

The agentic loop at ~line 964 correctly handles MCP tool calls — but _execute_with_parse_recovery throws before control returns there.

Expected Behavior

When the API response contains non-emit_output tool_use blocks (MCP tool calls), _execute_with_parse_recovery should return the response immediately so the agentic loop can:

1. Execute the MCP tool calls
2. Send results back to Claude
3. Continue the loop until emit_output is called

Actual Behavior

• Agent calls MCP tool (e.g., filesystem__read_file to read a plan file)
• _execute_with_parse_recovery doesn't recognize it as valid, enters recovery
• Recovery prompt confuses the model ("Your previous response did not contain valid JSON")
• All recovery attempts fail
• Workflow crashes: ProviderError: Failed to extract valid JSON after 2 recovery attempts

Reproduction

workflow:
  name: test
  entry_point: reader
  runtime:
    provider: claude
    mcp_servers:
      filesystem:
        command: npx
        args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
        tools: ["*"]
  input:
    path: { type: string, required: true }

agents:
  - name: reader
    model: claude-sonnet-4.6
    prompt: "Use read_file to read {{ workflow.input.path }} and summarize it"
    output:
      summary: { type: string }
      line_count: { type: integer }
    routes:
      - to: $end

echo "Hello World" > /tmp/test.txt
ANTHROPIC_API_KEY=sk-ant-... conductor run test.yaml --input path="/tmp/test.txt"

Error:

ProviderError: Claude API call failed: Failed to extract valid JSON after 2 recovery attempts

Suggested Fix

Add a check for non-emit_output tool_use blocks in _execute_with_parse_recovery, both after the initial response check and inside the recovery loop:

# After the emit_output check (~line 1180), add:
has_mcp_tool_use = any(
    hasattr(block, "type") and block.type == "tool_use" and block.name != "emit_output"
    for block in response.content
)
if has_mcp_tool_use:
    logger.debug("Response contains MCP tool calls, returning to agentic loop")
    return response

Same pattern inside the recovery loop after the _extract_structured_output check.

Relationship to #37

This bug was masked by #37 (empty tool_filter excludes all MCP tools). With tool_filter fixed and MCP tools actually reaching the API, this second bug prevents them from being executed. Both must be fixed for MCP tools to work with the Claude provider.

Environment

• Conductor v0.1.0 (commit 391ddaf)
• Python 3.12.13
• mcp SDK 1.26.0
• anthropic SDK 0.84.0

Impact

High — Even with #37 fixed, agents with output schemas cannot use MCP tools. The agentic tool loop exists and works correctly, but never gets to run because _execute_with_parse_recovery intercepts first.