[observability] Observability Coverage Report - 2026-04-25

[github-actions[bot]]

Executive Summary

Analyzed the latest 50 agentic workflow runs returned by the completed agenticworkflows logs --count 50 --start_date -7d call. Coverage percentages below use the 46 completed runs as the denominator; 4 in-progress runs are noted separately because their final artifacts may not have been uploaded yet.

Overall observability needs attention: firewall access logging coverage is 93%, and MCP telemetry coverage is 100%. Critical gaps are listed below with run links.

Key Alerts and Anomalies

🔴 Critical Issues:

• Smoke CI §24943301658: firewall enabled but access.log artifact was not found.

⚠️ Warnings:

• Smoke CI §24942995514: access.log present but contains 0 parsed Squid entries.
• Smoke CI §24942794438: access.log present but contains 0 parsed Squid entries.
• Daily Project Performance Summary Generator... §24943132993: MCP telemetry present but no tool calls were derived.
• Daily Safe Output Tool Optimizer §24943497502: run is still in_progress, so final artifact coverage is not evaluated yet.
• Daily Semgrep Scan §24943471949: run is still in_progress, so final artifact coverage is not evaluated yet.
• Daily Observability Report for AWF Firewall... §24943497698: run is still in_progress, so final artifact coverage is not evaluated yet.
• GitHub API Consumption Report Agent §24943263740: run is still in_progress, so final artifact coverage is not evaluated yet.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	15	14	93%	🔴
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	14	14	100%	⚠️

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Smoke CI	§24943301658	❌	0	0	0	🔴
Design Decision Gate 🏗️	§24943298811	✅	9	9	0	✅
Test Quality Sentinel	§24943298813	✅	12	12	0	✅
Design Decision Gate 🏗️	§24943246673	✅	9	9	0	✅
Test Quality Sentinel	§24943246674	✅	4	4	0	✅
Package Specification Librarian	§24943176268	✅	9	9	0	✅
Daily Project Performance Summary G...	§24943132993	✅	79	39	40	✅
Daily Skill Optimizer Improvements	§24943067618	✅	5	5	0	✅
Smoke CI	§24942995514	✅	0	0	0	⚠️
Smoke CI	§24942794438	✅	0	0	0	⚠️
Test Quality Sentinel	§24942774796	✅	4	4	0	✅
Test Quality Sentinel	§24942768716	✅	4	4	0	✅
Design Decision Gate 🏗️	§24942768749	✅	9	9	0	✅
Daily Safe Output Integrator	§24942745126	✅	13	7	6	✅
Design Decision Gate 🏗️	§24942774795	✅	10	10	0	✅

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Conclusion
Smoke CI	§24943301658	2026-04-25	failure

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Design Decision Gate 🏗️	§24943298811	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Test Quality Sentinel	§24943298813	rpc-messages.jsonl	2	safeoutputs	2	0	✅
Design Decision Gate 🏗️	§24943246673	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Test Quality Sentinel	§24943246674	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Package Specification Librarian	§24943176268	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Daily Project Performance Summary...	§24943132993	rpc-messages.jsonl	0	N/A	0	0	⚠️
Daily Skill Optimizer Improvements	§24943067618	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Smoke CI	§24942995514	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Smoke CI	§24942794438	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Test Quality Sentinel	§24942774796	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Test Quality Sentinel	§24942768716	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Design Decision Gate 🏗️	§24942768749	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Daily Safe Output Integrator	§24942745126	rpc-messages.jsonl	1	safeoutputs	1	0	✅
Design Decision Gate 🏗️	§24942774795	rpc-messages.jsonl	7	github, safeoutputs	7	0	✅

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

Workflow	Run ID	Date	Conclusion
None	N/A	N/A	N/A

In-Progress Runs Excluded From Coverage Denominators

Workflow	Run ID	Date	Status
Daily Safe Output Tool Optimizer	§24943497502	2026-04-25	in_progress
Daily Semgrep Scan	§24943471949	2026-04-25	in_progress
Daily Observability Report for AWF Firewa...	§24943497698	2026-04-25	in_progress
GitHub API Consumption Report Agent	§24943263740	2026-04-25	in_progress

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 167
• Allowed requests: 121
• Blocked requests: 46 (28%)
• Domains observed: 7 unique
• Most accessed domains: api.githubcopilot.com:443 (74), (unknown) (46), api.anthropic.com:443 (37), 151.101.64.223:443 (5), 151.101.128.223:443 (2)

Gateway Log Quality

• Telemetry source: rpc-messages.jsonl canonical fallback for completed MCP-enabled runs; preferred gateway.jsonl was not observed in this sample.
• Total MCP tool calls derived: 20
• MCP servers used: safeoutputs, github
• Error rate: 0.0%
• Average response time: N/A from available summary data; raw request/response pairing was not accessible for permission-restricted extracted log directories.

Additional Telemetry Artifacts

Artifact	Runs Present	Runs Analyzed
aw_info.json	19	50
agent-stdio.log	15	50
agent_output / agent_output.json	16	50
Safe output logs	16	50

Healthy Runs Summary

• Completed runs with complete required observability for their enabled components: 42 of 46
• Completed firewall-enabled runs with access.log: 14 of 15
• Completed MCP-enabled runs with gateway.jsonl or rpc-messages.jsonl: 14 of 14

Recommended Actions

1. Investigate the critical missing telemetry runs above before relying on those runs for incident debugging.
2. Consider adding or restoring mcp-logs/gateway.jsonl alongside rpc-messages.jsonl so preferred structured gateway metrics are available without reconstructing JSON-RPC exchanges.
3. Keep excluding in-progress runs from coverage gates until workflow artifacts are finalized, then re-evaluate them on the next scheduled report.

📊 Historical Trends

Historical trend data was not available in the downloaded log cache. This report should be compared with prior [observability] discussions for week-over-week movement.

References:

• Smoke CI: §24943301658
• Smoke CI: §24942995514
• Smoke CI: §24942794438

---

Analysis window: Last 7 days | Runs analyzed: 50 | Completed runs evaluated: 46

Warning

Firewall blocked 4 domains

The following domains were blocked by the firewall during workflow execution:

• ab.chatgpt.com
• api.github.com
• chatgpt.com
• github.com

💡 Tip: api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "api.github.com"
    - "chatgpt.com"
    - "github.com"

See Network Configuration for more information.

Generated by Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on Apr 26, 2026, 11:54 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #28645.