Add --parse support for firewall logs in logs and audit commands by Copilot · Pull Request #2349 · github/gh-aw

Copilot · 2025-10-25T00:19:48Z

The --parse flag now parses firewall logs in addition to agent logs, generating firewall.md alongside log.md.

Changes

Added parseFirewallLogs function in pkg/cli/logs.go
- Reuses embedded parse_firewall_logs.cjs script with mocked GitHub Actions environment
- Searches squid-logs/ or workflow-logs/squid-logs/ directories
- Generates markdown summary of blocked/allowed requests
- Gracefully skips when no firewall logs present
Integrated firewall parsing into logs and audit commands
- Both commands now parse firewall logs when --parse is used
- Success messages displayed for generated firewall.md
Updated help text to document firewall log parsing in both commands

Usage

# Parse both agent and firewall logs
gh aw audit 18795259023 --parse

# Output:
# ✓ Parsed log for run 18795259023 → logs/run-18795259023/log.md
# ✓ Parsed firewall logs for run 18795259023 → logs/run-18795259023/firewall.md

The generated firewall.md contains blocked domain statistics and request counts.

Original prompt

Updated command text:

Update the logs and audit commands with the --parse option to also parse the firewall logs using the JavaScript parser for the firewall logs.

If this is for a gh-aw agentic-workflows prompt, you might phrase the instruction more explicitly:

Update the logs and audit commands to accept a --parse option that enables parsing of firewall logs using the JavaScript firewall log parser. Ensure the parser is applied when --parse is provided.

Use this run to get a firewall log or run an audit https://github.com/githubnext/gh-aw/actions/runs/18795259023

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2025-10-25T00:49:38Z

Agentic Changeset Generator triggered by this pull request.

Copilot

Pull Request Overview

This PR adds firewall log parsing capability to the logs and audit commands via the --parse flag. When enabled, both agent and firewall logs are parsed, generating log.md and firewall.md respectively.

Key Changes:

Added parseFirewallLogs function that wraps an embedded JavaScript parser
Integrated firewall parsing into existing --parse flag workflows
Added comprehensive test coverage for firewall log parsing scenarios

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
pkg/cli/logs.go	Implements `parseFirewallLogs` function and integrates it into the logs command's parse workflow
pkg/cli/audit.go	Integrates firewall log parsing into the audit command's parse workflow
pkg/cli/logs_firewall_parse_test.go	Adds test coverage for firewall log parsing in various scenarios (with logs, without logs, empty directories)
docs/src/content/docs/reference/frontmatter-full.md	Removes outdated firewall feature flag documentation
.github/workflows/research.lock.yml	Updates firewall log parser with improved validation and filtered domain display

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pkg/cli/logs.go

.github/workflows/research.lock.yml

Copilot · 2025-10-25T00:51:00Z

.github/workflows/research.lock.yml

+
+              if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {
+
+                return null;


[nitpick] Multiple validation checks return null on failure without logging why parsing failed. Consider adding debug logging to indicate which field validation failed, as this would help diagnose parsing issues in production firewall logs.

Copilot · 2025-10-25T00:51:00Z

.github/workflows/research.lock.yml

+
+                return null;
+
+              }
+
+              const domain = fields[2];
+
+              if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {
+
+                return null;
+
+              }
+
+              const destIpPort = fields[3];
+
+              if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {
+
+                return null;
+
+              }
+
+              const status = fields[6];
+
+              if (status !== "-" && !/^\d+$/.test(status)) {
+
+                return null;
+
+              }
+
+              const decision = fields[7];
+
+              if (decision !== "-" && !decision.includes(":")) {
+
+                return null;
+


[nitpick] Multiple validation checks return null on failure without logging why parsing failed. Consider adding debug logging to indicate which field validation failed, as this would help diagnose parsing issues in production firewall logs.

Suggested change

return null;

}

const domain = fields[2];

if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {

return null;

}

const destIpPort = fields[3];

if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

return null;

console.debug(`Firewall log parse failed: invalid clientIpPort "${clientIpPort}"`);

return null;

}

const domain = fields[2];

if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {

console.debug(`Firewall log parse failed: invalid domain "${domain}"`);

return null;

}

const destIpPort = fields[3];

if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {

console.debug(`Firewall log parse failed: invalid destIpPort "${destIpPort}"`);

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

console.debug(`Firewall log parse failed: invalid status "${status}"`);

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

console.debug(`Firewall log parse failed: invalid decision "${decision}"`);

return null;

Copilot · 2025-10-25T00:51:00Z

.github/workflows/research.lock.yml

+
+                return null;
+
+              }
+
+              const domain = fields[2];
+
+              if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {
+
+                return null;
+
+              }
+
+              const destIpPort = fields[3];
+
+              if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {
+
+                return null;
+
+              }
+
+              const status = fields[6];
+
+              if (status !== "-" && !/^\d+$/.test(status)) {
+
+                return null;
+
+              }
+
+              const decision = fields[7];
+
+              if (decision !== "-" && !decision.includes(":")) {
+
+                return null;
+


[nitpick] Multiple validation checks return null on failure without logging why parsing failed. Consider adding debug logging to indicate which field validation failed, as this would help diagnose parsing issues in production firewall logs.

Suggested change

return null;

}

const domain = fields[2];

if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {

return null;

}

const destIpPort = fields[3];

if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

return null;

console.debug(`Firewall log parse failed: invalid clientIpPort "${clientIpPort}"`);

return null;

}

const domain = fields[2];

if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)) {

console.debug(`Firewall log parse failed: invalid domain "${domain}"`);

return null;

}

const destIpPort = fields[3];

if (destIpPort !== "-" && !/^[\d.]+:\d+$/.test(destIpPort)) {

console.debug(`Firewall log parse failed: invalid destIpPort "${destIpPort}"`);

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

console.debug(`Firewall log parse failed: invalid status "${status}"`);

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

console.debug(`Firewall log parse failed: invalid decision "${decision}"`);

return null;

Copilot · 2025-10-25T00:51:00Z

.github/workflows/research.lock.yml

+
+                return null;
+
+              }
+
+              const status = fields[6];
+
+              if (status !== "-" && !/^\d+$/.test(status)) {
+
+                return null;
+
+              }
+
+              const decision = fields[7];
+
+              if (decision !== "-" && !decision.includes(":")) {
+
+                return null;
+


[nitpick] Multiple validation checks return null on failure without logging why parsing failed. Consider adding debug logging to indicate which field validation failed, as this would help diagnose parsing issues in production firewall logs.

Suggested change

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

return null;

console.debug(`Parsing failed: destIpPort field invalid ('${destIpPort}'). Expected format: IP:port or '-'.`);

return null;

}

const status = fields[6];

if (status !== "-" && !/^\d+$/.test(status)) {

console.debug(`Parsing failed: status field invalid ('${status}'). Expected numeric or '-'.`);

return null;

}

const decision = fields[7];

if (decision !== "-" && !decision.includes(":")) {

console.debug(`Parsing failed: decision field invalid ('${decision}'). Expected to contain ':' or be '-'.`);

return null;

Copilot · 2025-10-25T00:51:01Z

pkg/cli/logs.go

+	jsScript := workflow.GetLogParserScript("parse_firewall_logs")
+	if jsScript == "" {
+		if verbose {
+			fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Failed to get firewall log parser script"))


The error message 'Failed to get firewall log parser script' doesn't explain why it failed or what the user should do. Consider adding context such as 'Firewall log parser script not found in embedded resources' or similar to help with troubleshooting.

Suggested change

fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Failed to get firewall log parser script"))

fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Firewall log parser script not found in embedded resources (parse_firewall_logs)"))

.github/workflows/research.lock.yml

…mands

Update CLI documentation to reflect recent feature additions: - Document firewall log parsing in --parse flag (PR #2349, #2350) - Logs and audit commands now generate firewall.md files - JSON output includes firewall analysis - Update --dependabot documentation (PR #2359) - Added pip and Go ecosystem support - Clarified command detection patterns - Add repository feature validation section (PR #2347) - Compile validates discussions/issues enabled - Prevents runtime failures for incompatible workflows 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Initial plan

b1799dd

Copilot AI assigned Copilot and pelikhan Oct 25, 2025

Copilot started work on behalf of pelikhan October 25, 2025 00:20 View session

Copilot AI and others added 5 commits October 25, 2025 00:25

Initial exploration: understand firewall log parsing structure

6ee017f

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Add parseFirewallLogs function to parse firewall logs with --parse flag

4715062

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Add comprehensive tests for parseFirewallLogs function

e384d65

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Update help text for --parse flag to mention firewall logs

fd3b608

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Add --parse support for firewall logs in logs and audit commands

2d6e310

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Update logs and audit commands to support --parse option~~ Add --parse support for firewall logs in logs and audit commands Oct 25, 2025

Copilot AI requested a review from pelikhan October 25, 2025 00:40

Copilot finished work on behalf of pelikhan October 25, 2025 00:40

pelikhan approved these changes Oct 25, 2025

View reviewed changes

pelikhan marked this pull request as ready for review October 25, 2025 00:49

Copilot AI review requested due to automatic review settings October 25, 2025 00:49

Copilot AI reviewed Oct 25, 2025

View reviewed changes

skip-ci: Add changeset for firewall log parsing in logs and audit com…

b4ef4c3

…mands

pelikhan merged commit 3e71eb1 into main Oct 25, 2025
4 checks passed

pelikhan deleted the copilot/update-logs-audit-commands branch October 25, 2025 01:08

github-actions bot mentioned this pull request Oct 25, 2025

[docs] Update CLI documentation for features from 2025-10-25 #2377

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --parse support for firewall logs in logs and audit commands#2349

Add --parse support for firewall logs in logs and audit commands#2349
pelikhan merged 7 commits intomainfrom
copilot/update-logs-audit-commands

Copilot AI commented Oct 25, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 25, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		if (domain !== "-" && !/^[a-zA-Z0-9]([a-zA-Z0-9-][a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-][a-zA-Z0-9])?)*:\d+$/.test(domain)) {

		return null;

	fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Failed to get firewall log parser script"))
	fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Firewall log parser script not found in embedded resources (parse_firewall_logs)"))

Conversation

Copilot AI commented Oct 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Usage

Uh oh!

github-actions bot commented Oct 25, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Oct 25, 2025 •

edited

Loading