Context
In pkg/cli/audit.go lines ~294-351, there's a repeated pattern where data extraction errors are only shown in --verbose mode:
missingTools, err := extractMissingToolsFromRun(...)
if err != nil && verbose { // Only shows if --verbose!
fmt.Fprintln(os.Stderr, console.FormatWarningMessage(...))
}
This applies to: firewall log parsing, MCP tool extraction, policy analysis, gateway metrics, access logs, and missing tools/data extraction.
When these fail silently, the user sees a clean report with missing sections and no indication that data was lost. "0 firewall blocks" and "firewall parsing failed" look identical.
Proposal
Add a DataCompleteness section to AuditData that always renders (regardless of verbose flag):
type DataCompleteness struct {
FirewallLogs DataSourceStatus // "ok", "missing", "parse_error"
MCPToolUsage DataSourceStatus
PolicyManifest DataSourceStatus
GatewayLogs DataSourceStatus
AgentOutput DataSourceStatus
AccessLogs DataSourceStatus
}
Collect errors during extraction and render a summary at the end of every audit report:
Data Sources:
✓ Firewall logs (142 requests parsed)
✓ MCP tool usage (12 calls from 3 servers)
✗ Policy manifest (file not found)
✓ Gateway logs (3 servers)
⚠ Agent output (parse warning: truncated JSON)
Why
An audit report that silently drops sections is worse than no report at all — it creates false confidence. Every audit report should be self-documenting about what it could and couldn't analyze.
Implementation
- Define
DataCompleteness struct with status + message per data source
- Thread it through each
extract* / analyze* call in audit.go
- Add to
AuditData and always render in renderConsole() (not gated by verbose)
- Include in JSON output as well
Parent epic: #22735
Context
In
pkg/cli/audit.golines ~294-351, there's a repeated pattern where data extraction errors are only shown in--verbosemode:This applies to: firewall log parsing, MCP tool extraction, policy analysis, gateway metrics, access logs, and missing tools/data extraction.
When these fail silently, the user sees a clean report with missing sections and no indication that data was lost. "0 firewall blocks" and "firewall parsing failed" look identical.
Proposal
Add a
DataCompletenesssection toAuditDatathat always renders (regardless of verbose flag):Collect errors during extraction and render a summary at the end of every audit report:
Why
An audit report that silently drops sections is worse than no report at all — it creates false confidence. Every audit report should be self-documenting about what it could and couldn't analyze.
Implementation
DataCompletenessstruct with status + message per data sourceextract*/analyze*call inaudit.goAuditDataand always render inrenderConsole()(not gated by verbose)Parent epic: #22735