[cli-tools-test] audit tool returns MCP isError for invalid run IDs instead of graceful JSON error

[github-actions]

Problem Description

The audit tool returns an MCP protocol error (isError=true) for an invalid or non-existent run ID, while the logs and compile tools both return graceful JSON error responses for similar invalid inputs. This inconsistency makes automated usage harder and causes confusing error messages.

Tool

• Tool: audit
• Command: agenticworkflows audit --run_id_or_url 9999999999

Steps to Reproduce

1. Call the audit MCP tool with a non-existent run ID (e.g., 9999999999)
2. Observe the response

Expected Behavior

A graceful JSON response like the other tools:

{
  "error": "Run ID 9999999999 not found",
  "run_id_or_url": "9999999999",
  "suggestions": ["Verify the run ID is correct", "Use the 'logs' tool to list recent run IDs"]
}

With exit code 0 and isError=false.

Actual Behavior

{"error":"failed to audit workflow run: ✗ failed to fetch run metadata","run_id_or_url":"9999999999"}
[error] [agenticworkflows] Tool returned isError=true: 101 chars

The CLI exits with an error code due to isError=true.

Comparison: Graceful errors in other tools

Tool	Invalid input	Behavior
logs	Non-existent workflow_name	Returns JSON with message field, isError=false ✅
compile	Non-existent workflow	Returns JSON array with valid: false and errors[], isError=false ✅
audit	Invalid run ID	Returns isError=true ❌

Environment

• Repository: github/gh-aw
• Run ID: 24873815037
• Date: 2026-04-24

Impact

• Severity: Low
• Frequency: Always (reproducible)
• Workaround: Catch the MCP protocol error in calling code

Additional Context

The logs tool already demonstrates the correct pattern: it returns {"runs": [], "message": "workflow '...' not found. Use the 'status' tool to see all available workflows..."} for non-existent workflow names. The audit tool should similarly return a structured JSON error instead of raising an MCP protocol error for a "not found" case.

Generated by Daily CLI Tools Exploratory Tester · ● 1.4M · ◷

• expires on May 1, 2026, 5:34 AM UTC