feat: add docs-preview-reviewer agentic workflow by Copilot · Pull Request #293 · github/gh-aw-firewall

Copilot · 2026-01-17T09:35:54Z

Adds an AI-powered documentation reviewer that triggers on PRs modifying docs, providing structured feedback with visual previews before merge.

Changes

New workflow .github/workflows/docs-preview-reviewer.md:
- Triggers on PRs touching docs/**, docs-site/**, **/*.md
- Builds the docs site locally and starts a preview server
- Uses Playwright to capture screenshots of rendered documentation pages
- Uploads screenshots as assets for visual preview
- Reviews for clarity, formatting, links, style consistency
- Posts single review comment with hide-older-comments: true to keep PR clean
Tools: github, bash, playwright
Permissions: Read-only (contents, pull-requests, issues)
Output: Uses safe-outputs.add-comment and safe-outputs.upload-asset for safe PR commenting with embedded screenshots

Example output format

## 📖 Documentation Review

### Overview
Brief summary of changes reviewed.

### 📸 Visual Preview
Screenshots of the rendered documentation pages (uploaded as assets):
![Page Screenshot](link-to-uploaded-screenshot.png)

### ✅ What Looks Good
- Positive aspects

### 💡 Suggestions for Improvement
- Actionable suggestions (if any)

Custom agent used: create-agentic-workflow
Design agentic workflows using GitHub Agentic Workflows (gh-aw) extension with interactive guidance on triggers, tools, and security best practices.

Original prompt

Create an agentic workflow to use #235

Custom agent used: create-agentic-workflow
Design agentic workflows using GitHub Agentic Workflows (gh-aw) extension with interactive guidance on triggers, tools, and security best practices.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

github-actions · 2026-01-17T09:46:59Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2026-01-17T09:47:00Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-01-17T09:47:11Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	77.19%	77.26%	📈 +0.07%
Statements	77.27%	77.34%	📈 +0.07%
Functions	77.17%	77.17%	➡️ +0.00%
Branches	69.76%	69.81%	📈 +0.05%

📁 Per-file Coverage Changes (2 files)

File	Lines (Before → After)	Statements (Before → After)
`src/domain-patterns.ts`	97.2% → 97.4% (+0.14%)	97.3% → 97.4% (+0.14%)
`src/ssl-bump.ts`	32.1% → 33.3% (+1.19%)	32.1% → 33.3% (+1.19%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

.github/workflows/docs-preview-reviewer.lock.yml

+      comment_repo: ""
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


.github/workflows/docs-preview-reviewer.lock.yml

+      output_types: ${{ steps.collect_output.outputs.output_types }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


.github/workflows/docs-preview-reviewer.lock.yml

+      total_count: ${{ steps.missing_tool.outputs.total_count }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


.github/workflows/docs-preview-reviewer.lock.yml

+      success: ${{ steps.parse_results.outputs.success }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


.github/workflows/docs-preview-reviewer.lock.yml

+      activated: ${{ steps.check_membership.outputs.is_team_member == 'true' }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


.github/workflows/docs-preview-reviewer.lock.yml

+      process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


github-actions · 2026-01-17T09:49:18Z

Smoke Test Results (Claude)

Last 2 merged PRs:

chore: remove smoke-codex workflow due to missing key #291: chore: remove smoke-codex workflow due to missing key
ci: add agentic documentation preview workflow with Playwright #246: ci: add agentic documentation preview workflow with Playwright

Test Results:

✅ GitHub MCP - fetched PR data
✅ Playwright - navigated to github.com, title contains "GitHub"
✅ File write - created test file
✅ Bash - verified file contents

Overall: PASS

AI generated by Smoke Claude

github-actions · 2026-01-17T09:49:26Z

Smoke Test Results (Copilot Engine)

Last 2 Merged PRs:

chore: remove smoke-codex workflow due to missing key #291: chore: remove smoke-codex workflow due to missing key
feat: port plan workflow from gh-aw repository #230: feat: port plan workflow from gh-aw repository

Test Results:

✅ GitHub MCP: Retrieved PR data successfully
❌ Playwright: Blocked by firewall (playwright.azureedge.net not allowlisted)
✅ File Writing: Created test file successfully
✅ Bash Tool: Verified file content

Overall: PARTIAL PASS (3/4 tests passed)

cc: @Mossaka

AI generated by Smoke Copilot

github-actions · 2026-01-17T09:49:41Z

📖 Documentation Review

Overview

This PR introduces a new agentic workflow (docs-preview-reviewer.md) that automatically reviews documentation changes in pull requests. The workflow is triggered when PRs modify documentation files and provides structured feedback on clarity, formatting, links, and style consistency.

✅ What Looks Good

Clear Purpose: The workflow description succinctly explains its function - reviewing docs and providing feedback before merge
Well-Structured Prompt: The agent prompt is organized into clear sections (Context, Task, Review Criteria, Output Format, Guidelines) that provide comprehensive instructions
Comprehensive Review Criteria: Covers all essential aspects - content quality, formatting, links, and style consistency
Thoughtful Configuration:
- hide-older-comments: true prevents PR clutter by collapsing previous reviews
- timeout-minutes: 10 provides reasonable execution time
- Appropriate GitHub permissions (read-only for contents, issues, pull-requests)
Helpful Guidelines: The closing guidelines emphasize constructive, helpful feedback rather than nitpicking - this sets the right tone for automated reviews
Proper Path Triggers: Correctly targets documentation files (docs/**, docs-site/**, **/*.md)

💡 Suggestions for Improvement

Add Examples Section: Consider adding a "Before/After Examples" or "Common Issues" section to help the AI recognize typical documentation problems specific to this repository (e.g., firewall domain whitelist formatting, code block language tags for shell commands)
Reference Existing Style Guides: If this repository has specific documentation conventions (e.g., in CONTRIBUTING.md or AGENTS.md), reference them explicitly in the prompt so the AI can check consistency
Consider Security Context: Since this reviews potentially untrusted PR content from public contributors, you might want to add a note about treating PR descriptions/comments as untrusted data (though the current XPIA protection in my system prompt covers this)

📝 Minor Notes

The workflow looks production-ready and follows GitHub Agentic Workflows best practices
The output format template with emojis (📖 ✅ 💡 📝) is user-friendly and makes reviews scannable
The balance between being thorough and avoiding over-criticism is well-calibrated

This review was generated by the Documentation Preview Reviewer workflow.

AI generated by Documentation Preview Reviewer

Mossaka

@copilot address the comments

Mossaka · 2026-01-17T20:09:07Z

.github/workflows/docs-preview-reviewer.md

+tools:
+  github:
+    toolsets: [default]
+safe-outputs:


Add playwright tool and in the instruction, tell the agent to use playwright tool to upload screenshots of the docs site.

Added playwright tool with allowed_domains: [localhost] and upload-asset safe output for screenshots. Updated the instructions to build the docs site locally, capture screenshots of rendered pages, and embed them in the review comment. Commit: 1956f9c

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

.github/workflows/docs-preview-reviewer.lock.yml

+      published_count: ${{ steps.upload_assets.outputs.published_count }}
+    steps:
+      - name: Setup Scripts
+        uses: githubnext/gh-aw/actions/setup@v0.36.0


github-actions · 2026-01-17T20:36:45Z

🔒 Security Review - Critical Issue Found

I've reviewed PR #293 and identified a critical security vulnerability that completely bypasses the AWF firewall protection.

🚨 Critical Issue: Playwright MCP Server Bypasses Firewall

File: .github/workflows/docs-preview-reviewer.lock.yml (line 364)

Problematic Code:

"playwright": {
  "type": "local",
  "command": "docker",
  "args": ["run", "-i", "--rm", "--init", "--network", "host", "mcr.microsoft.com/playwright/mcp", ...],
  "tools": ["*"]
}

Security Issue: The Playwright MCP server is configured with --network host, which gives the container direct access to the host's network stack, completely bypassing all AWF firewall restrictions.

Why This Is Critical:

Firewall Bypass: The --network host flag removes Docker's network isolation and allows the Playwright container to access ANY network endpoint on the internet, regardless of the --allow-domains configuration
Defeats AWF Purpose: The entire purpose of AWF is to restrict agent network access to approved domains. This configuration undermines that security boundary
Attack Vector: A malicious or compromised AI agent could use Playwright tools to exfiltrate data, scan internal networks, or access arbitrary external services
Unrestricted Access: While the AWF agent is restricted to specific domains (github.com, npmjs.org, etc.), the Playwright container can reach anywhere

How It Bypasses the Firewall:

The AWF firewall applies iptables rules to the awf-agent container's network namespace
--network host makes Playwright run in the host's network namespace, not the container's
Therefore, Playwright bypasses ALL iptables rules that AWF applies
Only the --allowed-hosts restriction in Playwright's own configuration provides protection (localhost only), but this is insufficient

Evidence from Workflow:

Line 719: AWF is configured with specific allowed domains
Line 364: Playwright MCP runs with --network host (bypasses AWF entirely)
Line 1265-1267 (markdown source): The network.allowed section suggests Playwright is intended to have restricted access, but the --network host flag contradicts this

⚠️ Suggested Actions

Immediate Fix: Remove --network host from the Playwright configuration
Alternative Approaches:
- Use Docker's default bridge network (like other MCP servers)
- If Playwright needs to access localhost services (like the preview server), use host.docker.internal instead
- Consider running Playwright in the same network as the docs preview server
Security Validation: After fixing, verify that Playwright can only access localhost preview servers and cannot reach external domains not in the allowlist

📋 No Other Critical Security Issues Found

I reviewed the remaining configuration and found:

✅ Domain allowlist is appropriately scoped (no wildcards or overly broad patterns)
✅ Shell command restrictions are reasonable (npm ci, npm run build, npm run preview*)
✅ No dangerous container capabilities added (no privileged, SYS_ADMIN, etc.)
✅ GitHub MCP server properly configured with read-only mode and lockdown
✅ Proper use of AWF version v0.8.2

Recommendation: This PR should not be merged until the --network host issue is resolved. The current configuration creates a significant security vulnerability that defeats the purpose of the Agentic Workflow Firewall.

AI generated by Security Guard

Initial plan

ec30fc6

Copilot AI assigned Copilot and Mossaka Jan 17, 2026

Copilot started work on behalf of Mossaka January 17, 2026 09:36 View session

Copilot AI and others added 2 commits January 17, 2026 09:39

feat: create docs-preview-reviewer agentic workflow for issue #235

69fe9d5

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

fix: address code review feedback for docs-preview-reviewer workflow

e93eafa

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Create agentic workflow for issue 235~~ feat: add docs-preview-reviewer agentic workflow Jan 17, 2026

Copilot AI requested a review from Mossaka January 17, 2026 09:42

Copilot finished work on behalf of Mossaka January 17, 2026 09:42

Merge branch 'main' into copilot/create-agentic-workflow

73a2750

github-advanced-security bot found potential problems Jan 17, 2026

View reviewed changes

github-actions bot added the smoke-claude label Jan 17, 2026

Mossaka reviewed Jan 17, 2026

View reviewed changes

Copilot started work on behalf of Mossaka January 17, 2026 20:23 View session

feat: add playwright tool for docs site screenshots

1956f9c

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

Copilot finished work on behalf of Mossaka January 17, 2026 20:27

Copilot AI requested a review from Mossaka January 17, 2026 20:27

Mossaka marked this pull request as ready for review January 17, 2026 20:32

github-advanced-security bot found potential problems Jan 17, 2026

View reviewed changes

Mossaka closed this Jan 17, 2026

Conversation

Copilot AI commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example output format

Uh oh!

github-actions bot commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Check warning

Uh oh!

Check warning

Uh oh!

Check warning

Uh oh!

Check warning

Uh oh!

Check warning

Uh oh!

Check warning

Uh oh!

github-actions bot commented Jan 17, 2026

Smoke Test Results (Claude)

Uh oh!

github-actions bot commented Jan 17, 2026

Smoke Test Results (Copilot Engine)

Uh oh!

github-actions bot commented Jan 17, 2026

📖 Documentation Review

Overview

✅ What Looks Good

💡 Suggestions for Improvement

📝 Minor Notes

Uh oh!

Mossaka left a comment

Choose a reason for hiding this comment

Uh oh!

Mossaka Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

Check warning

Uh oh!

github-actions bot commented Jan 17, 2026

🔒 Security Review - Critical Issue Found

🚨 Critical Issue: Playwright MCP Server Bypasses Firewall

⚠️ Suggested Actions

📋 No Other Critical Security Issues Found

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 17, 2026 •

edited

Loading

github-actions bot commented Jan 17, 2026 •

edited

Loading

github-actions bot commented Jan 17, 2026 •

edited

Loading

github-actions bot commented Jan 17, 2026 •

edited

Loading