Skip to content

Add daily Claude workflow for cross-repo gh-aw compilation compatibility checks#26802

Merged
pelikhan merged 3 commits intomainfrom
copilot/create-agentic-workflow-daily-compilation-check
Apr 17, 2026
Merged

Add daily Claude workflow for cross-repo gh-aw compilation compatibility checks#26802
pelikhan merged 3 commits intomainfrom
copilot/create-agentic-workflow-daily-compilation-check

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 17, 2026

Summary

  • Added a new workflow: .github/workflows/daily-aw-cross-repo-compile-check.md
  • Compiled lock file: .github/workflows/daily-aw-cross-repo-compile-check.lock.yml

What it does

  • Runs daily (weekdays) + manual dispatch
  • Uses Claude engine
  • Discovers repositories that appear to use gh-aw by searching for .github/workflows/*.lock.yml patterns
  • Ranks by stars, selects top 20, shallow-clones each, and runs compile/fix/recompile checks with latest local gh-aw build
  • Persists run artifacts and rolling history in cache-memory
  • Clusters failures and creates actionable compatibility issues (including likely missing codemods and recurring syntax/config problems)

Safety and reporting

  • Uses safe-outputs for issue creation
  • Disables mentions/backlinks in generated reports
  • Includes missing-tool reporting path

Validation

  • gh aw compile .github/workflows/daily-aw-cross-repo-compile-check.md
  • make agent-finish ❌ (pre-existing timeout in pkg/cli tests: TestFetchIncludeFromSource_SectionExtraction, unchanged by this workflow-only PR)
  • parallel_validation:
    • Code Review ✅ (addressed reported items)
    • CodeQL scan timed out (tool reported timeout)

@pelikhan pelikhan marked this pull request as ready for review April 17, 2026 04:50
Copilot AI review requested due to automatic review settings April 17, 2026 04:50
@pelikhan pelikhan merged commit 596c5c8 into main Apr 17, 2026
57 of 103 checks passed
@pelikhan pelikhan deleted the copilot/create-agentic-workflow-daily-compilation-check branch April 17, 2026 04:50
@github-actions github-actions Bot mentioned this pull request Apr 17, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new daily Agentic Workflows (gh-aw) Claude-based workflow to discover popular external repos using gh-aw lockfiles and run cross-repo compile/fix/recompile compatibility checks, caching results and filing actionable issues.

Changes:

  • Introduces a new Claude workflow definition for daily cross-repo compatibility auditing.
  • Adds the compiled lock workflow YAML produced by gh aw compile.
Show a summary per file
File Description
.github/workflows/daily-aw-cross-repo-compile-check.md Defines the agent mission, discovery strategy, per-repo compile/fix flow, caching, and issue reporting rules.
.github/workflows/daily-aw-cross-repo-compile-check.lock.yml Compiled GitHub Actions workflow that executes the agent with safe-outputs, caching, and reporting plumbing.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 4

if [ ! -d "$REPO_DIR/.git" ]; then
# Record clone failure object in results.jsonl before continuing.
# Include: repository, stars, clone_status=failed, probable_root_cause, top_error_lines.
exit 0
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the clone-failure branch, the snippet uses exit 0. If this code is used inside a loop over repositories, exit will terminate the entire run on the first clone failure rather than moving on to the next repo (the preceding comment says “before continuing”). Use continue (in-loop) or return from a per-repo function instead, after appending the failure record.

Suggested change
exit 0
continue

Copilot uses AI. Check for mistakes.

## Phase 3: Analyze Failures for Compatibility Gaps

From repositories with `compile_after_status != success`, perform clustering by error signature.
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compile_after_status is defined above as a numeric exit code, but Phase 3 filters failures using compile_after_status != success. That condition doesn’t match the documented type and will be ambiguous for the agent. Compare against an exit code (e.g., non-zero) or store a separate boolean field (e.g., compile_after_ok).

Suggested change
From repositories with `compile_after_status != success`, perform clustering by error signature.
From repositories with non-zero `compile_after_status`, perform clustering by error signature.

Copilot uses AI. Check for mistakes.
Comment on lines +91 to +92
GH_AW_BIN="gh aw"
gh aw --version | tee "$RUN_DIR/gh-aw-version.txt"
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GH_AW_BIN is set to the string "gh aw", but later invoked as "$GH_AW_BIN" compile ... / fix .... In bash this will try to execute a binary literally named gh aw (with a space) and will fail. Use gh as the command and pass aw as an argument (or use a bash array like GH_AW_CMD=(gh aw) and invoke it correctly) so the fallback path actually works.

Suggested change
GH_AW_BIN="gh aw"
gh aw --version | tee "$RUN_DIR/gh-aw-version.txt"
GH_AW_BIN="$RUN_DIR/gh-aw-fallback"
cat > "$GH_AW_BIN" <<'EOF'
#!/usr/bin/env bash
exec gh aw "$@"
EOF
chmod +x "$GH_AW_BIN"
"$GH_AW_BIN" --version | tee "$RUN_DIR/gh-aw-version.txt"

Copilot uses AI. Check for mistakes.
GH_AW_BIN="gh aw"
gh aw --version | tee "$RUN_DIR/gh-aw-version.txt"
else
echo "missing gh-aw binary and gh aw fallback" > "$RUN_DIR/tooling-error.txt"
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the missing-tooling path, the script exits immediately (exit 1) after writing a text file. That conflicts with the workflow’s own requirement to always emit a safe-output call, and it also means the missing_tool handler likely won’t have any structured item to report. Emit a missing_tool safe output (and/or a report_incomplete/noop as appropriate) before exiting so the failure is surfaced consistently.

Suggested change
echo "missing gh-aw binary and gh aw fallback" > "$RUN_DIR/tooling-error.txt"
echo "missing gh-aw binary and gh aw fallback" > "$RUN_DIR/tooling-error.txt"
safe-output missing_tool <<JSON
{"tool":"gh-aw","message":"missing gh-aw binary and gh aw fallback","details_file":"$RUN_DIR/tooling-error.txt"}
JSON

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants