feat: pipeline orchestrator v3 + docs + daily test-analysis#144
Merged
feat: pipeline orchestrator v3 + docs + daily test-analysis#144
Conversation
- Changelog: added entries for PRs #140-#143, dependabot bumps, orchestrator v1/v2, PR #113 auto-merge success - Agentic-workflows: replaced removed orchestrator section with current bash implementation, updated agent inventory, added history entry - Test-analysis: changed cron from weekly to daily (0 9 * * *) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds three capabilities to the pipeline orchestrator: 1. Cron trigger (*/15 * * * *) — orchestrator runs automatically 2. Issue dispatch — finds oldest aw-labeled issue without aw-dispatched or aw-stuck labels, dispatches implementer 3. Review loop management — after resolving threads, removes review-response-attempted label to re-enable responder for next Copilot review round. Counter labels (review-response-N) cap at 3 rounds before marking PR as aw-stuck. Also: aw-stuck label makes orchestrator ignore stuck PRs so they don't block new issue dispatch. Permissions updated to issues:write and actions:write for label management and workflow dispatch. Part of #135 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds “v3” functionality to the bash-based pipeline orchestrator workflow and updates docs/changelog plus the test-analysis schedule to reflect current operations.
Changes:
- Extend
pipeline-orchestrator.ymlwith a 15-minute cron trigger, issue dispatch when no activeawPR exists, and review-loop label management. - Update documentation/changelog to describe the orchestrator and recent workflow evolution.
- Change
test-analysisschedule from weekly to daily and recompile its lock file.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
docs/changelog.md |
Adds an entry summarizing recent orchestrator/quality-gate related PRs. |
docs/agentic-workflows.md |
Updates orchestrator documentation, workflow inventory, and historical notes. |
.github/workflows/test-analysis.md |
Updates gh-aw workflow frontmatter schedule to daily. |
.github/workflows/test-analysis.lock.yml |
Compiler output reflecting the new daily cron schedule. |
.github/workflows/pipeline-orchestrator.yml |
Implements orchestrator v3: cron, issue dispatch gating, and review-loop management. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+214
to
+232
| # Review loop management | ||
| HAS_RESPONSE_LABEL=$(echo "$LABELS" | jq -r 'if index("review-response-attempted") then "yes" else "" end') | ||
|
|
||
| if [[ -n "$HAS_RESPONSE_LABEL" ]]; then | ||
| # Count existing review-response-N labels | ||
| ROUND=$(echo "$LABELS" | jq -r '[.[] | select(startswith("review-response-"))] | length') | ||
|
|
||
| if [[ "$ROUND" -ge 3 ]]; then | ||
| # Max rounds reached — mark as stuck | ||
| echo " ⚠️ PR #${PR}: Review-response loop hit 3 rounds. Marking as stuck." | ||
| gh pr edit "$PR" --add-label "aw-stuck" | ||
| gh pr comment "$PR" --body "⚠️ Pipeline orchestrator: review-response loop reached 3 rounds without resolution. Marking as stuck for human review." | ||
| else | ||
| # Remove review-response-attempted to re-enable responder | ||
| NEXT_ROUND=$((ROUND + 1)) | ||
| echo " Enabling review round ${NEXT_ROUND} for PR #${PR}" | ||
| gh pr edit "$PR" --remove-label "review-response-attempted" | ||
| gh pr edit "$PR" --add-label "review-response-${NEXT_ROUND}" | ||
| fi |
Comment on lines
189
to
+195
| and (.comments.nodes[-1].author?.login // "" | . != "copilot-pull-request-reviewer") | ||
| )] | .[].id') | ||
|
|
||
| if [[ -z "$RESOLVABLE" ]]; then | ||
| echo "PR #${PR}: No threads to resolve." | ||
| echo "::endgroup::" | ||
| continue | ||
| if [[ -n "$RESOLVABLE" ]]; then | ||
| COUNT=0 | ||
| for THREAD_ID in $RESOLVABLE; do | ||
| echo " Resolving thread: ${THREAD_ID}" |
Comment on lines
+133
to
+138
| # Add aw-dispatched label to prevent re-dispatch | ||
| gh issue edit "$ISSUE_NUM" --add-label "aw-dispatched" | ||
|
|
||
| # Dispatch the implementer | ||
| gh workflow run issue-implementer.lock.yml -f issue_number="$ISSUE_NUM" | ||
|
|
Comment on lines
+395
to
+408
| **Triggers:** | ||
| - `workflow_run`: after Review Responder completes | ||
| - `push: branches: [main]`: after merges (rebases stale PRs) | ||
| - `schedule: */15 * * * *`: cron (issue dispatch, cleanup) — planned for v3 | ||
| - `workflow_dispatch`: manual with optional `pr_number` input | ||
|
|
||
| **Current state**: Issue dispatch (implementer) and CI fixer dispatch are inactive until #135 is implemented. Review-responder and quality-gate continue to work normally. | ||
| **Capabilities (current):** | ||
| - **v1 — Thread resolution**: Queries review threads via GraphQL, resolves unresolved threads where the last commenter is not `copilot-pull-request-reviewer` (meaning someone addressed the comment) | ||
| - **v2 — Auto-rebase**: Detects PRs behind main via `mergeStateStatus: BEHIND`, rebases onto `origin/main`, force-pushes with lease. On conflict: adds `aw-needs-rebase` label + comment for human | ||
|
|
||
| ### Review Responder Thread ID Lookup | ||
| **Planned:** | ||
| - **v3** — Issue dispatch (find `aw` issues, dispatch implementer), cron trigger, review-fix loop management | ||
| - **v4** — CI fixer dispatch | ||
| - **v5** — Stale PR cleanup |
| @@ -1,7 +1,7 @@ | |||
| --- | |||
| # Weekly test suite analysis | |||
… detects noop Responder: moved review-response-attempted label to after confirming unresolved threads exist. Previously the label was added on noop paths (e.g. quality gate approval trigger), locking out the responder from addressing actual Copilot review comments. Orchestrator: detects when review-response-attempted label is present but all unresolved threads are still from copilot-pull-request-reviewer (nobody addressed them). Removes the label so the responder can retry. Discovered during live test: quality gate approval triggered responder, responder noop'd (no comments), but still got the label. Part of #135 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
microsasa
force-pushed
the
docs/orchestrator-v1-v2-updates
branch
from
8853155 to
748bb0e
Compare
There was a problem hiding this comment.
Pull request overview
Updates the repo’s agentic workflow automation by extending the bash-based pipeline orchestrator to v3 (cron-driven issue dispatch + review-loop handling), refreshing documentation/changelog entries, and increasing test-analysis cadence to daily.
Changes:
- Add orchestrator v3 capabilities: 15-min cron trigger, oldest-issue dispatch when no active
awPRs, and review-loop label management withaw-stuckhandling. - Update workflow docs/changelog to reflect recent orchestrator/quality-gate work and workflow inventory changes.
- Change
test-analysisfrom weekly to daily and recompile the lock workflow.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
docs/changelog.md |
Adds an entry summarizing today’s orchestrator/quality-gate/dependabot changes. |
docs/agentic-workflows.md |
Updates orchestrator section + workflow inventory, but needs v3 status wording aligned with the actual workflow changes. |
.github/workflows/test-analysis.md |
Changes schedule to daily cron (frontmatter still says “Weekly”). |
.github/workflows/test-analysis.lock.yml |
Compiler output reflecting the new daily cron schedule. |
.github/workflows/review-responder.md |
Refines responder loop-guard instructions and when to apply review-response-attempted. |
.github/workflows/pipeline-orchestrator.yml |
Implements v3 cron/dispatch/review-loop features and introduces aw-stuck gating. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+401
to
+408
| **Capabilities (current):** | ||
| - **v1 — Thread resolution**: Queries review threads via GraphQL, resolves unresolved threads where the last commenter is not `copilot-pull-request-reviewer` (meaning someone addressed the comment) | ||
| - **v2 — Auto-rebase**: Detects PRs behind main via `mergeStateStatus: BEHIND`, rebases onto `origin/main`, force-pushes with lease. On conflict: adds `aw-needs-rebase` label + comment for human | ||
|
|
||
| ### Review Responder Thread ID Lookup | ||
| **Planned:** | ||
| - **v3** — Issue dispatch (find `aw` issues, dispatch implementer), cron trigger, review-fix loop management | ||
| - **v4** — CI fixer dispatch | ||
| - **v5** — Stale PR cleanup |
| @@ -1,7 +1,7 @@ | |||
| --- | |||
| # Weekly test suite analysis | |||
Comment on lines
+50
to
+52
| 2. Check the review that triggered this workflow. If the review has no comments (e.g., a plain approval with no inline comments), stop — there is nothing to address. Do NOT add any labels when stopping here. | ||
|
|
||
| 3. Check if the PR already has the label `review-response-attempted`. If it does, add a comment to the PR saying "Review response already attempted — stopping to prevent loops. Manual intervention needed." and stop. | ||
| 3. Read the unresolved review comment threads on the PR (not just the latest review — get all unresolved threads). If there are no unresolved threads, stop — there is nothing to address. Do NOT add any labels when stopping here. |
| gh pr edit "$PR" --remove-label "review-response-attempted" | ||
| else | ||
| # Count existing review-response-N labels | ||
| ROUND=$(echo "$LABELS" | jq -r '[.[] | select(startswith("review-response-"))] | length') |
| nodes { number } | ||
| nodes { | ||
| number | ||
| labels(first: 10) { nodes { name } } |
| nodes { | ||
| number | ||
| title | ||
| labels(first: 10) { nodes { name } } |
Comment on lines
+133
to
+138
| # Add aw-dispatched label to prevent re-dispatch | ||
| gh issue edit "$ISSUE_NUM" --add-label "aw-dispatched" | ||
|
|
||
| # Dispatch the implementer | ||
| gh workflow run issue-implementer.lock.yml -f issue_number="$ISSUE_NUM" | ||
|
|
Comment on lines
+395
to
+399
| **Triggers:** | ||
| - `workflow_run`: after Review Responder completes | ||
| - `push: branches: [main]`: after merges (rebases stale PRs) | ||
| - `schedule: */15 * * * *`: cron (issue dispatch, cleanup) — planned for v3 | ||
| - `workflow_dispatch`: manual with optional `pr_number` input |
Responder: step 2 now checks for existing unresolved threads even when the triggering review has no comments. Previously a clean re-review (e.g. from Copilot re-request) would noop even with unaddressed threads from a prior review. Orchestrator: requests Copilot re-review when unaddressed threads exist and no review-response-attempted label is present. This triggers the event chain: Copilot review → responder → orchestrator. Also requests re-review when removing a stale response label. Discovered during live test: quality gate approval triggered responder noop, leaving 4 Copilot threads permanently unaddressed. Part of #135 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Pipeline orchestrator v3
New capabilities added to
pipeline-orchestrator.yml:1. Cron trigger (
*/15 * * * *)Orchestrator runs automatically every 15 minutes.
2. Issue dispatch
When no active
awPR exists, finds the oldestaw-labeled issue (withoutaw-dispatchedoraw-stucklabels) and dispatches the implementer.3. Review loop management
After resolving threads, removes
review-response-attemptedlabel to re-enable the responder for the next Copilot review round. Counter labels (review-response-1,review-response-2,review-response-3) track iterations. After 3 rounds, marks PR asaw-stuck.4. aw-stuck label
Stuck PRs are ignored by the dispatch gate — the orchestrator moves on to the next issue instead of blocking.
Docs
docs/agentic-workflows.mdupdated: orchestrator section, agent inventory, historydocs/auto_pr_orchestrator_aw.mduntouched (archived)Test-analysis daily
Changed cron from weekly to daily (
0 9 * * *). Recompiled lock file.Related issues