feat: PolyPilot CI integration test workflow (Linux/GTK + MauiDevFlow)

[PureWeen]

Goal

A GitHub Action that builds and runs PolyPilot on Linux/GTK with MauiDevFlow for automated UI inspection and end-to-end testing.

Architecture

ubuntu-latest → Build PolyPilot.Gtk (Debug) → xvfb → MauiDevFlow agent → smoke tests

Current State

• Workflow file created
• Build passes in CI
• App launches under xvfb
• MauiDevFlow connects
• Smoke tests pass
• Copilot CLI integration
• End-to-end scenario

DO NOT MERGE — work in progress

[github-actions]

Expert Code Review — PR #615

Methodology: 3 independent reviewers with adversarial consensus. Findings required 2/3+ agreement. Single-reviewer findings were challenged via follow-up and included only if 2+ ultimately agreed.

Findings Summary

#	Severity	Consensus	Issue
1	🟡 MODERATE	3/3	Binary name mismatch (PolyPilot.Gtk vs PolyPilot) + missing empty-binary guard
2	🟡 MODERATE	2/3	Unquoted ${{ }} expression in run: block — breakage on empty/spaces
3	🟡 MODERATE	3/3	`
4	🟡 MODERATE	2/3	Wrong MAUI workload (maui-android instead of maui for GTK target)
5	🟡 MODERATE	3/3	All smoke tests swallow failures — workflow always reports green
6	🟡 MODERATE	3/3	No permissions: block — inherits excessive write-all token scope
7	🟢 MINOR	3/3	issue_number and scenario inputs declared but never used
8	🟢 MINOR	2/3	Stale action versions (@v4 vs @v6/@v7 used in build.yml)

Key Interaction Chain

Findings 1, 2, and 3 compound into a complete workflow failure: the binary name mismatch (#1) guarantees the primary find always misses → the missing guard lets an empty value through → the unquoted expression (#2) silently backgrounds nothing → MauiDevFlow wait times out 60s later with a misleading error. Meanwhile, the wrong workload (#4) may fail the build entirely, and || true on tool install (#3) ensures that failure is also invisible.

Even after fixing these correctness issues, the swallowed smoke test failures (#5) mean the workflow would report false-positive green runs, defeating the purpose of integration testing.

Discarded Findings (no consensus)

• /tmp screenshot path — 1/3; correctly identified as standard CI runner practice, not applicable to the agent-workspace /tmp prohibition
• Wrong package ID (Microsoft.Maui.Cli) — 1/3; verified incorrect — Microsoft.Maui.Cli is the correct package that wraps maui devflow
• Manual-only trigger — 1/3; follow-up reviewers unanimously disagreed — workflow_dispatch is intentional for heavyweight integration tests

CI Status

• No CI checks have run on this PR (new workflow file only — no build targets to validate)

Test Coverage

• N/A — this is a workflow file, not application code. No unit tests affected.

Prior Reviews

• No prior human reviews found on this PR.

Generated by Expert Code Review (auto) for issue #615

[github-actions]

🟡 MODERATE — Binary name mismatch: primary search always fails (Flagged by: 3/3 reviewers)

PolyPilot.Gtk.csproj declares (AssemblyName)PolyPilot(/AssemblyName), so the output binary is named PolyPilot, not PolyPilot.Gtk. This find will never match. The fallback on line 64 (PolyPilot*) is overly broad and may match .dll, .runtimeconfig.json, or other artifacts.

If both searches fail, BINARY is empty and line 79 becomes a bare & — a shell no-op that assigns a transient PID. kill -0 succeeds momentarily (subshell still in process table), printing a false "✅ PolyPilot is running" before failing 60s later at MauiDevFlow wait with a misleading error.

Suggested fix:

BINARY=$(find PolyPilot.Gtk/bin/Debug -name "PolyPilot" -type f -executable | head -1)
if [ -z "$BINARY" ]; then
  echo "❌ No executable binary found"; exit 1
fi

[github-actions]

🟡 MODERATE — Unquoted expression interpolation (Flagged by: 2/3 reviewers)

GitHub Actions substitutes $\{\{ steps.find-binary.outputs.binary }} at YAML parse time, before the shell sees it. If the resolved path is empty (see binary name finding above) or contains spaces, the shell splits it incorrectly. An empty value makes this line & — backgrounding nothing but assigning a PID that briefly appears valid.

Suggested fix: Pass through an environment variable:

env:
  BINARY: $\{\{ steps.find-binary.outputs.binary }}
run: |
  if [ -z "$BINARY" ]; then echo "❌ No binary"; exit 1; fi
  "$BINARY" &

[github-actions]

🟡 MODERATE — || true silently swallows tool install failure (Flagged by: 3/3 reviewers)

If the MauiDevFlow CLI install fails (feed outage, version conflict, network error), the step exits 0 and the workflow proceeds. All subsequent maui devflow commands fail with "command not found" after burning 60s on the wait timeout, with a misleading "agent did not connect" error.

Suggested fix: Remove || true and add a verification:

dotnet tool install --global Microsoft.Maui.Cli \
  --add-source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json \
  --prerelease
command -v maui || { echo "❌ MauiDevFlow CLI not installed"; exit 1; }

[github-actions]

🟡 MODERATE — Wrong MAUI workload for GTK/Linux target (Flagged by: 2/3 reviewers)

maui-android installs only Android SDK tooling. The GTK project targets net10.0 (plain .NET) and needs the base MAUI workload. The existing build.yml uses dotnet workload install maui for its non-Android jobs.

Suggested fix:

run: dotnet workload install maui

[github-actions]

🟡 MODERATE — Smoke tests swallow all failures; CI always reports green (Flagged by: 3/3 reviewers)

Every smoke test uses || echo "... failed" or || true, exiting 0 regardless. A completely broken app produces a green checkmark. The only step that can fail the job is the MauiDevFlow wait — and that indicates infrastructure failure, not behavioral regression.

Suggested fix: Use continue-on-error: true at the step level for informational steps (logs) but let core validation steps (MAUI status, visual tree) fail the workflow:

- name: "Smoke Test: MAUI status"
  run: |
    export DISPLAY=:99
    maui devflow MAUI status

- name: "Smoke Test: Application logs"
  continue-on-error: true
  run: |
    export DISPLAY=:99
    timeout 5 maui devflow MAUI logs || true

[github-actions]

🟡 MODERATE — No permissions: block; inherits excessive token scope (Flagged by: 3/3 reviewers after follow-up)

Every job in the existing build.yml declares an explicit permissions: block. This workflow has none, inheriting the repository default (write-all for workflow_dispatch). The workflow only needs read access.

Suggested fix:

jobs:
  integration-test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    permissions:
      contents: read

[github-actions]

🟢 MINOR — Unused issue_number and scenario inputs (Flagged by: 3/3 reviewers)

Both inputs are declared but never referenced in any step. A user selecting scenario: full gets the same smoke tests — silently misleading.

Suggested fix: Remove both inputs until implemented, or wire scenario into conditional steps with if: inputs.scenario == 'full'.

[github-actions]

🟢 MINOR — Stale action versions (Flagged by: 2/3 reviewers)

build.yml uses actions/checkout@v6 and actions/upload-artifact@v7. This workflow uses @v4 for both. Align with existing repo conventions.

Suggested fix: actions/checkout@v6 (line 26), actions/upload-artifact@v7 (line 141).

[PureWeen]

🔍 Code Review — PR #615

feat: PolyPilot CI integration test workflow
CI Status: ✅ All 3 checks passing (Linux, Mac Catalyst, Windows)

---

All 8 findings from Round 1 — FIXED ✅

#	Finding	Status
1	Missing if guard on CDP step	✅ Fixed
2	Windows artifact path mismatch	✅ Fixed
3	sleep 30 → poll loop	✅ Fixed
4	Unnecessary token on CDP step	✅ Fixed
5	or-true swallowing failures	✅ Fixed
6	Fragile cdp_eval interpolation	✅ Fixed
7	Empty binary guard	✅ Fixed
8	Hardcoded Xcode	✅ Fixed

Verified Clean

• Token security ✅
• GTK DI registrations ✅
• CDP injection safety ✅
• Cleanup ✅

Recommendation

✅ Approve — all findings addressed, CI green on all 3 platforms.