fix(compliance-audit): eliminate false positives + apply API-based fixes#120
fix(compliance-audit): eliminate false positives + apply API-based fixes#120
Conversation
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 23 minutes and 54 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughUpdated the compliance audit script to parse Dependabot ecosystems and labels with quote-flexible regex, adjust ecosystem block extraction, exempt reusable Changes
Sequence Diagram(s)(omitted — changes are script logic tweaks within a single component) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@don-petry — this PR fixes the false positives in the compliance audit script identified in issue #119, plus applies API-based fixes (CodeQL setup and rulesets). Please review and merge when ready. |
There was a problem hiding this comment.
Pull request overview
This PR updates the org compliance audit script to reduce false positives and align checks with how the standards are applied across repositories.
Changes:
- Relaxed Dependabot YAML matching to accept single-quoted ecosystem names and labels.
- Updated workflow permissions auditing to skip
workflow_call-only reusable workflows. - Expanded
AGENTS.mdorg-reference detection to accept GitHub blob-style URLs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@scripts/compliance-audit.sh`:
- Around line 793-797: The grep regex that checks $decoded currently uses
"petry-projects/\.github/blob/[^/]+/AGENTS\.md" which rejects branch names
containing slashes; update the pattern used in the if test to allow any
characters (including slashes) between "blob/" and "/AGENTS.md" (for example
replace the "[^/]+” segment with ".+"), i.e. adjust the grep -qE pattern that
references $decoded so it matches "petry-projects/\.github/blob/.+/AGENTS\.md"
while keeping the existing "\.github/AGENTS\.md" alternative.
- Around line 489-494: Replace the brittle on: extraction and whitelist logic
(the on_triggers variable computed from decoded and the subsequent grep checks)
with a simple filename-based check: detect reusable workflows by testing the
workflow file name for the suffix -reusable.yml or -reusable.yaml (i.e., if
filename ends with -reusable.yml or -reusable.yaml then treat as reusable and
continue), and remove the awk/grep on_triggers logic that inspects decoded to
avoid inline syntax and missing-trigger edge cases. Ensure you use the existing
loop's filename variable (the variable used to hold the current workflow file)
in the new check and delete the on_triggers and related grep blocks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: d2c157b5-1dce-42ee-bcac-c465efa4a097
📒 Files selected for processing (1)
scripts/compliance-audit.sh
- Dependabot YAML quote style: grep patterns now accept both
single-quoted ('github-actions') and double-quoted ("github-actions")
YAML values. YAML allows either; the old patterns required double
quotes, flagging valid single-quoted configs as non-compliant.
- Workflow permissions: reusable workflows (workflow_call-only triggers)
are now skipped. Their permissions are set by the caller, so requiring
a top-level permissions: block in *-reusable.yml files was a false
positive that blocked every org-level reusable workflow.
- AGENTS.md org reference: accept GitHub blob URLs
(petry-projects/.github/blob/<branch>/AGENTS.md) in addition to the
canonical path format (.github/AGENTS.md in link text). Both forms
unambiguously point to the org-level standards file.
Closes #119 (partial — API-based fixes applied separately)
Co-authored-by: don-petry <don-petry@users.noreply.github.com>
- Replace brittle awk/grep reusable-workflow detection with filename convention check (*-reusable.yml), avoiding false skips from unlisted triggers or inline on: syntax - Fix blob URL regex to allow slashed branch names (blob/.+/AGENTS.md instead of blob/[^/]+/AGENTS.md) - Align comment with what the regex actually matches Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@scripts/compliance-audit.sh`:
- Around line 249-256: The awk block extraction uses an unquoted pattern that
can falsely match substrings (e.g., "pnpm" when eco=npm); update the awk pattern
used in the block assignment (the command that builds variable `block` using awk
"/package-ecosystem:.*$eco/{...}") to require the ecosystem name to be quoted
the same way as the earlier grep (match either double-quoted or single-quoted
"$eco"/'$eco') and ensure the exit condition `!/$eco/` is changed to use the
same quoted-aware match so the block stops at the next ecosystem entry
correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 6b30d569-1452-40c6-b9b3-ae85c7e3a683
📒 Files selected for processing (1)
scripts/compliance-audit.sh
don-petry
force-pushed
the
claude/issue-119-20260409-1218
branch
from
745825a to
4b906a5
Compare
…traction The awk pattern used unquoted $eco which caused "npm" to substring-match "pnpm", potentially spanning the extracted block into the wrong ecosystem entry. Align awk with the existing grep by requiring quoted values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Summary
Addresses all actionable findings from the 2026-04-09 compliance audit (issue #119).
Changes in this PR
scripts/compliance-audit.sh— three false-positive fixes:Dependabot YAML quote style — old grep patterns required
"double-quoted"YAML values for ecosystem names and labels; YAML permits both styles. Patterns updated to accept'single-quoted'as well. This resolves themissing-github-actions-ecosystem,missing-security-label, andmissing-dependencies-labelfindings forgoogle-app-scripts(whosedependabot.ymluses single quotes).Reusable workflow permissions — the permissions check now skips
workflow_call-only workflows (*-reusable.yml). Reusable workflows inherit permissions from their caller, so requiring a top-levelpermissions:block in them was incorrect. This resolves the fourmissing-permissions-*-reusable.ymlfindings for the.githubrepo.AGENTS.md org reference — the check now accepts GitHub blob URLs (
petry-projects/.github/blob/<branch>/AGENTS.md) in addition to the canonical path format (.github/AGENTS.mdin link text). Both unambiguously point to the org-level standards file. This resolvesagents-md-missing-org-refforContentTwinandmarkets.API-based fixes applied directly (no file changes required)
These were applied via the GitHub API as part of this remediation and will show as compliant in the next audit run:
markets,google-app-scripts,ContentTwin,broodly,bmad-bgreat-suite,TalkTermpr-qualityruleset created.github,google-app-scriptscode-qualityruleset created.github,google-app-scriptsRemaining open items (require
GH_PAT_WORKFLOWSsecret or manual action)The following findings affect
.github/workflows/*.ymlfiles in various repos. GitHub App tokens cannot write to workflow files without theworkflowsscope (GH_PAT_WORKFLOWSsecret). These must be fixed manually or via a PAT-enabled run:unpinned-actions-*.ymlacross all reposcodeql.ymlfiles — these per-repo workflow files should be removed now that GitHub-managed CodeQL default setup is enabled; the checkstray-codeql-workflowwill clear once removedci.yml—TalkTermneeds a CI pipeline addedrequired-claude-check-broken—TalkTermruleset references a check that no longer existsCloses #119
Generated with Claude Code
Summary by CodeRabbit