Conversation
github-actions
bot
added
the
big_boom
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
|
||
| ### Triage Reports (When Root Cause Is Unclear) | ||
|
|
||
| When a **clear repeated failure pattern** exists (3+ occurrences across different runs) but the root cause cannot be definitively identified from available CI logs alone, file a **triage report** instead of a full investigation. A triage report: |
There was a problem hiding this comment.
[MEDIUM] Triage instruction conflicts with issue-format gate
This new section asks the agent to file a triage report when a flaky pattern repeats but root cause is unclear, but the Issue Format later still says to not file an issue unless root cause is identified. In that scenario (3+ repeats, no confirmed root cause), the workflow now gives contradictory directions and can suppress the intended triage issue output.
Please align the Issue Format with triage mode (for example, allow a triage report with hypotheses/investigation steps when root cause is not yet proven).
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (35)
📝 WalkthroughWalkthroughThis PR updates many gh-aw workflows and fragments: adds a code-quality-audit fragment and a Code Quality Audit workflow; introduces a public Possibly related PRs
✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Align flaky triage output rules, make shared audit severity/schema guidance importer-driven, and extend project-summary bot exclusions to stale and threshold logic. Made-with: Cursor
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Restore valid workflow_dispatch defaults for agentics-maintenance, add fail-fast severity-threshold validation, and inject explicit threshold policy guidance for test-coverage detection prompts. Made-with: Cursor
There was a problem hiding this comment.
Requesting changes due to a workflow contract gap: severity-threshold is newly exposed in two workflows but not validated before use.
- Medium:
.github/workflows/gh-aw-code-duplication-detector.md:59— accepts arbitrary strings forseverity-thresholdand forwards them into prompt behavior; typos (e.g.meduim) silently degrade severity gating. - Medium:
.github/workflows/gh-aw-framework-best-practices.md:49— same unvalidatedseverity-thresholdcontract; behavior becomes non-deterministic when callers pass invalid values.
Please add the same fail-fast validation used in gh-aw-code-quality-audit.md and gh-aw-test-coverage-detector.md (high|medium|low) to both workflows.
What is this? | From workflow: Trigger PR Review
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
.github/workflows/gh-aw-code-quality-audit.lock.yml (1)
165-170:⚠️ Potential issue | 🟡 MinorValidate
severity-thresholdbeforeactivation.Lines 165-170 interpolate the raw input into the prompt, but the guard does not run until Lines 666-669. Invalid values still consume the activation job and produce an artifact, so this is not actually fail-fast yet.
Based on learnings,
.lock.ymlfiles in this repo are generated bygh aw compile; make this change in.github/workflows/gh-aw-code-quality-audit.mdand recompile.Also applies to: 666-669
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/gh-aw-code-quality-audit.lock.yml around lines 165 - 170, The prompt currently interpolates the raw inputs (GH_AW_EXPR_08E89BCD / inputs.severity-threshold) into GH_AW_PROMPT before the guard/validation runs, so invalid values still trigger activation and artifact creation; move the severity-threshold validation to run before any use of GH_AW_EXPR_08E89BCD (and before the activation/job that creates GH_AW_PROMPT and GH_AW_SAFE_OUTPUTS), implement the guard in the source template .github/workflows/gh-aw-code-quality-audit.md (not the generated .lock.yml), and then recompile with `gh aw compile` so the compiled .lock.yml has the fail-fast check (also apply the same fix where GH_AW_EXPR_08E89BCD is used again around the activation block at the later occurrence).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/gh-aw-code-quality-audit.lock.yml:
- Around line 314-318: Update the serialization that builds
previous-findings.json to include recency fields so the agent can detect
recently closed issues: extend the object that currently emits only number,
title and state to also include closedAt and updatedAt (or equivalent
timestamps) when collecting issue data for previous-findings.json; make this
change in the source template that generates the workflow (the template that
produces the lines serializing number/title/state) and then recompile the
workflow using gh aw compile so the .lock.yml includes those new fields. Ensure
the agent logic that checks for "recently closed" uses the new
closedAt/updatedAt fields to decide when to noop versus file.
- Around line 378-379: The CI currently sets GH_AW_NOOP_REPORT_AS_ISSUE="true",
which turns expected noop runs into issues; update the audit template source to
set GH_AW_NOOP_REPORT_AS_ISSUE to "false" (and ensure the noop handling logic
still treats noop as the normal non-issue path) and recompile the generated
lockfile with the gh aw compile command so the .lock.yml reflects the change;
also search the template for other GH_AW_NOOP_REPORT_AS_ISSUE occurrences and
the noop handling block and make the same adjustment.
In @.github/workflows/gh-aw-code-quality-audit.md:
- Around line 30-33: The workflow currently marks inputs like
additional-instructions and title-prefix as required but doesn’t prevent callers
from passing empty or whitespace-only values; add an explicit validation step
(e.g., an initial job/step named validate_inputs) that checks trimmed values of
inputs.additional-instructions, inputs.title-prefix (and
inputs.severity-threshold if applicable) and fails early if any are empty or
only whitespace, returning a clear error message; reference these exact input
names (additional-instructions, title-prefix, severity-threshold) and perform
the check using a small run script or expression that trims the value and exits
non-zero when empty so downstream jobs are never executed with blank core
inputs.
In @.github/workflows/gh-aw-test-coverage-detector.lock.yml:
- Around line 706-709: The current "Validate severity threshold" step (checking
SEVERITY_THRESHOLD via the case ... esac) runs too late; move that validation to
an earlier location so it executes before any expensive setup (checkout,
runtime/tool install, repo-convention fetch, previous-findings lookup). Update
the source template `.github/workflows/gh-aw-test-coverage-detector.md` (not the
generated .lock.yml) to either add a new pre-step at the top of the job or a
separate lightweight job that validates SEVERITY_THRESHOLD using the same
case-check logic and fails fast on invalid values; keep the same variable name
SEVERITY_THRESHOLD and error message so behavior remains identical. Ensure the
new validation runs before steps that reference the repo or install tools
(checkout, setup-*, repo-convention, previous-findings) to avoid wasted runner
time.
---
Duplicate comments:
In @.github/workflows/gh-aw-code-quality-audit.lock.yml:
- Around line 165-170: The prompt currently interpolates the raw inputs
(GH_AW_EXPR_08E89BCD / inputs.severity-threshold) into GH_AW_PROMPT before the
guard/validation runs, so invalid values still trigger activation and artifact
creation; move the severity-threshold validation to run before any use of
GH_AW_EXPR_08E89BCD (and before the activation/job that creates GH_AW_PROMPT and
GH_AW_SAFE_OUTPUTS), implement the guard in the source template
.github/workflows/gh-aw-code-quality-audit.md (not the generated .lock.yml), and
then recompile with `gh aw compile` so the compiled .lock.yml has the fail-fast
check (also apply the same fix where GH_AW_EXPR_08E89BCD is used again around
the activation block at the later occurrence).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: cb846043-2d29-4ef1-9646-9f38406bef96
📒 Files selected for processing (5)
.github/workflows/agentics-maintenance.yml.github/workflows/gh-aw-code-quality-audit.lock.yml.github/workflows/gh-aw-code-quality-audit.md.github/workflows/gh-aw-test-coverage-detector.lock.yml.github/workflows/gh-aw-test-coverage-detector.md
| Before filing a new issue, check `/tmp/previous-findings.json` for issues this agent has already filed. | ||
|
|
||
| - Run `cat /tmp/previous-findings.json` to read the list of previously filed issue numbers and titles. | ||
| - If your finding closely matches an open or recently-closed issue in that list, call `noop` instead of filing a duplicate. | ||
| - Only file a new issue when the finding is genuinely distinct from all previous findings. |
There was a problem hiding this comment.
previous-findings.json needs recency fields.
Lines 314-318 tell the agent to suppress duplicates only for open or recently closed issues, but Lines 664-665 only serialize number, title, and state. Without closedAt or updatedAt, the agent cannot distinguish a fresh duplicate from a long-closed regression.
Based on learnings, .lock.yml files in this repo are generated by gh aw compile; make this change in .github/workflows/gh-aw-code-quality-audit.md and recompile.
Also applies to: 664-665
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/gh-aw-code-quality-audit.lock.yml around lines 314 - 318,
Update the serialization that builds previous-findings.json to include recency
fields so the agent can detect recently closed issues: extend the object that
currently emits only number, title and state to also include closedAt and
updatedAt (or equivalent timestamps) when collecting issue data for
previous-findings.json; make this change in the source template that generates
the workflow (the template that produces the lines serializing
number/title/state) and then recompile the workflow using gh aw compile so the
.lock.yml includes those new fields. Ensure the agent logic that checks for
"recently closed" uses the new closedAt/updatedAt fields to decide when to noop
versus file.
| If zero findings pass all four criteria, call `noop` with a brief reason and stop. **Noop is the expected outcome most days.** Filing nothing is a success when there is nothing worth filing. | ||
|
|
There was a problem hiding this comment.
Don't turn noop into an issue.
The prompt treats noop as the expected success path when nothing material is found, but Line 1456 sets GH_AW_NOOP_REPORT_AS_ISSUE to "true". That converts clean runs into issue noise and weakens the one-issue audit signal.
Based on learnings, .lock.yml files in this repo are generated by gh aw compile; make this change in .github/workflows/gh-aw-code-quality-audit.md and recompile.
Also applies to: 1447-1457
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/gh-aw-code-quality-audit.lock.yml around lines 378 - 379,
The CI currently sets GH_AW_NOOP_REPORT_AS_ISSUE="true", which turns expected
noop runs into issues; update the audit template source to set
GH_AW_NOOP_REPORT_AS_ISSUE to "false" (and ensure the noop handling logic still
treats noop as the normal non-issue path) and recompile the generated lockfile
with the gh aw compile command so the .lock.yml reflects the change; also search
the template for other GH_AW_NOOP_REPORT_AS_ISSUE occurrences and the noop
handling block and make the same adjustment.
| additional-instructions: | ||
| description: "Domain-specific audit criteria — what to look for, what to skip, and how to evaluate findings. This is the core of the audit and should define the categories, severity standards, and evidence expectations for the specific quality dimension being audited." | ||
| type: string | ||
| required: true |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Does GitHub Actions workflow_callwithrequired: true for string inputs still allow callers to pass an empty string? Please check the official GitHub Actions documentation for reusable workflow input semantics.
💡 Result:
Yes. In GitHub Actions reusable workflows (on.workflow_call.inputs), required: true means the caller must supply the input, but it does not mean the string must be non-empty.
GitHub’s workflow syntax docs state that if a default isn’t set, the default for a string input is "" (empty string) [1]. The same docs define required semantics as “whether the input must be supplied” (i.e., presence), not “must be non-empty” [2]. So a caller can satisfy a required string input by passing "", and the called workflow will receive "".
Sources (official GitHub Docs):
[1] GitHub Docs – Workflow syntax (on.workflow_call.inputs default for string is ""): (docs.github.com)
[2] GitHub Docs – Workflow syntax (inputs.<input_id>.required: “must be supplied”): (docs.github.com)
Citations:
- 1: https://docs.github.com/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions
- 2: https://docs.github.com/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions
Validate non-empty core inputs, not just severity-threshold.
In GitHub Actions reusable workflows, required: true enforces input presence only—callers can pass an empty string and satisfy the requirement. A blank additional-instructions removes the audit criteria entirely, and a blank title-prefix breaks the duplicate-search/title logic. Validate that these inputs are non-empty and non-whitespace.
Suggested hardening
- name: Validate severity threshold
env:
+ ADDITIONAL_INSTRUCTIONS: ${{ inputs.additional-instructions }}
SEVERITY_THRESHOLD: ${{ inputs.severity-threshold }}
+ TITLE_PREFIX: ${{ inputs.title-prefix }}
run: |
+ if [ -z "${ADDITIONAL_INSTRUCTIONS//[[:space:]]/}" ]; then
+ echo "additional-instructions must be non-empty." >&2
+ exit 1
+ fi
+ if [ -z "${TITLE_PREFIX//[[:space:]]/}" ]; then
+ echo "title-prefix must be non-empty." >&2
+ exit 1
+ fi
case "$SEVERITY_THRESHOLD" in
high|medium|low) ;;
*)
- echo "Invalid severity-threshold: '$SEVERITY_THRESHOLD'. Expected one of: high, medium, low."
+ echo "Invalid severity-threshold: '$SEVERITY_THRESHOLD'. Expected one of: high, medium, low." >&2
exit 1
;;
esacApplies to lines 30–33, 49–57, and 88–98.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/gh-aw-code-quality-audit.md around lines 30 - 33, The
workflow currently marks inputs like additional-instructions and title-prefix as
required but doesn’t prevent callers from passing empty or whitespace-only
values; add an explicit validation step (e.g., an initial job/step named
validate_inputs) that checks trimmed values of inputs.additional-instructions,
inputs.title-prefix (and inputs.severity-threshold if applicable) and fails
early if any are empty or only whitespace, returning a clear error message;
reference these exact input names (additional-instructions, title-prefix,
severity-threshold) and perform the check using a small run script or expression
that trims the value and exits non-zero when empty so downstream jobs are never
executed with blank core inputs.
| - env: | ||
| SEVERITY_THRESHOLD: ${{ inputs.severity-threshold }} | ||
| name: Validate severity threshold | ||
| run: "case \"$SEVERITY_THRESHOLD\" in\n high|medium|low) ;;\n *)\n echo \"Invalid severity-threshold: '$SEVERITY_THRESHOLD'. Expected one of: high, medium, low.\"\n exit 1\n ;;\nesac\n" |
There was a problem hiding this comment.
Move threshold validation before expensive setup.
At Line 706, invalid severity-threshold values still get past checkout, runtime setup, repo-convention fetch, and previous-findings lookup before the run fails. That makes this a late guard rather than a fail-fast one, and it wastes runner time on bad inputs. Put the validation in an earlier step/job before repository and tool setup starts.
Based on learnings, .lock.yml files in this repo are generated by gh aw compile, so make this change in .github/workflows/gh-aw-test-coverage-detector.md.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/gh-aw-test-coverage-detector.lock.yml around lines 706 -
709, The current "Validate severity threshold" step (checking SEVERITY_THRESHOLD
via the case ... esac) runs too late; move that validation to an earlier
location so it executes before any expensive setup (checkout, runtime/tool
install, repo-convention fetch, previous-findings lookup). Update the source
template `.github/workflows/gh-aw-test-coverage-detector.md` (not the generated
.lock.yml) to either add a new pre-step at the top of the job or a separate
lightweight job that validates SEVERITY_THRESHOLD using the same case-check
logic and fails fast on invalid values; keep the same variable name
SEVERITY_THRESHOLD and error message so behavior remains identical. Ensure the
new validation runs before steps that reference the repo or install tools
(checkout, setup-*, repo-convention, previous-findings) to avoid wasted runner
time.
Teach update-pr-body prompts to detect and remove repeated runtime footer blocks before rewriting PR descriptions, and document the dedup behavior in the workflow README. Made-with: Cursor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/gh-aw-update-pr-body.md (1)
83-83:⚠️ Potential issue | 🔴 CriticalMissing
messages-footerinput declaration.
inputs.messages-footeris referenced at line 83 (safe-outputs config) and line 101 (prompt context), but no corresponding input is declared in theinputs:section (lines 20-60). Callers cannot customize the footer since the input isn't exposed.🔧 Proposed fix: add the missing input
Add after line 60 (before
secrets:):default: "none" + messages-footer: + description: "Custom footer text appended to PR body updates. Defaults to a standard runtime-managed message." + type: string + required: false + default: "" secrets: COPILOT_GITHUB_TOKEN:Also applies to: 101-101
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/gh-aw-update-pr-body.md at line 83, Add a missing inputs declaration for "messages-footer" so callers can override the footer referenced by inputs.messages-footer; update the workflow's inputs: block to include a new input named messages-footer (with a sensible default and description) so the safe-outputs/footer interpolation and prompt context (where inputs.messages-footer is used) work correctly; ensure the new input is added alongside the existing inputs: entries (before the secrets: section) and uses the exact key "messages-footer" so references in the template and prompt resolve.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In @.github/workflows/gh-aw-update-pr-body.md:
- Line 83: Add a missing inputs declaration for "messages-footer" so callers can
override the footer referenced by inputs.messages-footer; update the workflow's
inputs: block to include a new input named messages-footer (with a sensible
default and description) so the safe-outputs/footer interpolation and prompt
context (where inputs.messages-footer is used) work correctly; ensure the new
input is added alongside the existing inputs: entries (before the secrets:
section) and uses the exact key "messages-footer" so references in the template
and prompt resolve.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5fc63b40-b9cd-4d76-b1a8-1991bf80c2a4
📒 Files selected for processing (3)
.github/workflows/gh-aw-update-pr-body.lock.yml.github/workflows/gh-aw-update-pr-body.mdgh-agent-workflows/update-pr-body/README.md
✅ Files skipped from review due to trivial changes (1)
- gh-agent-workflows/update-pr-body/README.md
|
|
||
| ```text | ||
| --- | ||
| The body of this PR is automatically managed by the workflow runtime. |
There was a problem hiding this comment.
[MEDIUM] Default footer dedup string does not match runtime footer
The dedup step says to remove the default footer shown above, but that default text ("The body of this PR is automatically managed by the workflow runtime.") does not match the actual configured runtime footer at .github/workflows/gh-aw-update-pr-body.md:83 (linked ela.st message with workflow/run URL).
When no messages-footer override is provided, the existing runtime footer won't match this dedup key, so repeated runs can keep appending another runtime footer instead of normalizing to one.
Please make the documented "default runtime footer text" exactly match the configured footer template (or dedupe using a stable marker that matches the runtime-generated footer).
1 participant
Summary
This PR further aligns GitHub Agent workflow prompts, safe-output flows, and shared audit behavior.
.github/workflows/gh-aw-code-quality-audit.md.additional-instructions.severity-threshold(high/medium/low) with explicit validation..github/workflows/gh-aw-fragments/code-quality-audit.md.severity-threshold:.github/workflows/gh-aw-code-duplication-detector.md.github/workflows/gh-aw-framework-best-practices.md.github/workflows/gh-aw-test-coverage-detector.mdgh-aw-test-coverage-detector.mdwith threshold validation and explicit severity-policy text..github/workflows/gh-aw-create-pr-from-issue.mdand generated lockfile.ready_to_make_prbeforecreate_pull_requestin issue/PR-creation flows.ready_to_push_to_prand stricter review-thread resolution/completion guidance in PR-fix/review flows.gh-aw-pr-actions-detective.mdwith duplicate-report suppression and a structured response format.gh-aw-bug-hunter.md(stronger definition of impactful bugs)gh-aw-flaky-test-investigator.md(triage-report path when root cause is unclear)gh-aw-performance-profiler.md(required before/after data-integrity checks)gh-aw-project-summary.md(shared bot identity filtering and stale-item exclusions)gh-aw-refactor-opportunist.md(explicit prioritization scoring with reuse bonus)gh-aw-text-auditor.md(noop for isolated cosmetic-only findings).github/workflows/gh-aw-update-pr-body.mdnow instructs runtime-footer normalization/deduplication before rewriting.gh-agent-workflows/update-pr-body/README.mddocuments runtime-footer deduplication behavior.agentics-maintenance.ymlmanual input handling by usingnoneas the default/no-op operation..lock.ymlworkflow files to keep metadata in sync.Intent
Improve consistency and signal quality across scheduled and on-demand auditing workflows by introducing shared quality-audit rules, tightening PR safety/push gates, and enabling issue-to-PR remediation chaining where appropriate.
The body of this PR is automatically managed by the Trigger Update PR Body workflow.