Skip to content

fix: prevent security review from re-posting comments on resolved threads #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: prevent security review from re-posting comments on resolved threads #80

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d7cb014
Initial plan
Copilot Feb 19, 2026
b03ec1a
fix: prevent security review from re-posting comments on resolved thr…
Copilot Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions .github/workflows/security-review.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,24 +42,37 @@ agent instructions.
`/tmp/gh-aw/cache-memory/security-review-pr-${{ github.event.pull_request.number }}.json`. It can contain information about previous review findings, categories, files reviewed, and timestamps for this PR.
- Identify recurring security patterns in this repository from
`/tmp/gh-aw/cache-memory/security-review-patterns.json`. It can contain information about recurring security issues and patterns in the repository.
- Avoid repeating the same inline comments from previous reviews if the previous comment is not resolved yet nor outdated (e.g., if the same issue is still present in the code or if the code has not changed since the last review).

2. **Fetch the pull request diff.** Read the pull request details and all
2. **Check existing review threads.** Before posting any new comment, use
the `pull_requests` toolset to fetch all existing review comment threads
for PR #${{ github.event.pull_request.number }}. For each thread:
- Record the file path, line number, and body of the original comment.
- Record whether the thread is **resolved** (i.e., marked as resolved by
a reviewer or via the `/create-issue` command).
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instruction states that threads can be "marked as resolved by a reviewer or via the /create-issue command", but it's unclear whether the /create-issue command actually resolves threads in the GitHub API. Based on the create-issue-command.md workflow, the /create-issue command creates an issue and replies to the comment, but there's no indication it marks the thread as resolved. This could lead to confusion for the AI agent executing this workflow. Consider clarifying this by either: (1) verifying that /create-issue does resolve threads and keeping this language, (2) removing the reference to /create-issue if it doesn't actually resolve threads, or (3) instructing the agent to also check if a thread has an associated issue created via /create-issue as an alternative indicator that it's been addressed.

Suggested change
a reviewer or via the `/create-issue` command).
a reviewer).

Copilot uses AI. Check for mistakes.
- **Do not re-post a comment** for any finding that already has a
resolved thread on the same file and line. Even if the finding is still
technically present in the code, do not re-open resolved threads by
posting a duplicate comment.
- **Do not re-post a comment** for any finding that already has an
unresolved thread on the same file and line and whose code has not
changed since the last review.
Comment on lines +57 to +58
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instruction to skip re-posting for unresolved threads "whose code has not changed since the last review" is ambiguous and may be difficult for the AI agent to execute reliably. The workflow doesn't specify how the agent should determine whether the code at a specific line has changed between reviews. Consider providing more explicit guidance, such as: (1) instructing the agent to compare the current line's content with what was flagged in the previous review (stored in cache memory), (2) checking the commit history for that specific line, or (3) comparing the diff hunk context from the previous review comment with the current state. Without clearer implementation guidance, this instruction may not prevent duplicate comments as intended.

Suggested change
unresolved thread on the same file and line and whose code has not
changed since the last review.
unresolved thread on the same file and line **if the underlying code is
unchanged since the last review**. To determine whether the code has
changed:
- Compare the current line content and its immediate diff hunk context
(for example, the line plus approximately three surrounding lines) with
what was previously recorded for that thread in cache memory at
`/tmp/gh-aw/cache-memory/security-review-pr-${{ github.event.pull_request.number }}.json`.
- If the cached content for that file path and line (including its diff
hunk context) matches the current diff hunk, treat the code as
unchanged and skip posting a new comment on that thread.
- If no prior cache entry exists, or if the content or context differs,
treat the code as changed and apply your normal commenting logic.

Copilot uses AI. Check for mistakes.

3. **Fetch the pull request diff.** Read the pull request details and all
changed files for PR #${{ github.event.pull_request.number }}.

3. **Review every changed file** against all 15 security posture categories
4. **Review every changed file** against all 15 security posture categories
from the imported agent instructions. Focus only on the lines that were
added or modified in the diff — do not flag pre-existing code that was not
touched.

4. **Post inline review comments** on specific code lines where you find
5. **Post inline review comments** on specific code lines where you find
security issues. Each comment must include:
- The security category (e.g., "Input Validation", "Secrets")
- Severity: critical, high, medium, low, or informational
- A clear description of the issue and why it matters
- A concrete, actionable recommendation or code fix

5. **Submit the review.** After posting all inline comments:
6. **Submit the review.** After posting all inline comments:
- If you found any **critical** or **high** severity issues, submit the
review with `REQUEST_CHANGES` and a summary body listing the top findings.
- If you found only **medium** or **low** issues, submit with `COMMENT` and
Expand All @@ -75,7 +88,7 @@ agent instructions.
have been resolved or are outdated, submit with `COMMENT` and include
a summary of what was resolved.

6. **Update memory.** After submitting the review:
7. **Update memory.** After submitting the review:
- Write/update PR-specific memory at
`/tmp/gh-aw/cache-memory/security-review-pr-${{ github.event.pull_request.number }}.json`
including review timestamp, findings summary, categories found, and files
Expand Down
Loading