feat: add AgentShield CI standard and agent-shield.yml workflow template#25
Merged
feat: add AgentShield CI standard and agent-shield.yml workflow template#25
Conversation
Defines agent configuration security requirements (no secrets, no permission bypasses, valid frontmatter, org references) and provides a reusable workflow template for all repos. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 12 minutes and 43 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds repository agent standards documentation and a new AgentShield GitHub Actions workflow; updates CI standards to require and document the AgentShield workflow that validates CLAUDE.md, AGENTS.md, SKILL.md frontmatter, secret/credential absence, and permission-bypass rules. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
rect rgba(200,230,255,0.5)
Actor Repo
end
rect rgba(200,255,200,0.5)
Participant Actions as "GitHub Actions\n(AgentShield)"
end
rect rgba(255,230,200,0.5)
Participant Scanner as "AgentShield\nscan/validators"
end
Repo->>Actions: push / pull_request to main
Actions->>Scanner: run checks on files (`CLAUDE.md`, `AGENTS.md`, `SKILL.md`, .claude/*, etc.)
Scanner->>Repo: read files (contents, YAML frontmatter)
Scanner->>Scanner: validate cross-references, scan secrets, detect permission-bypass, validate SKILL frontmatter
Scanner-->>Actions: report results (pass/warn/fail)
Actions->>Repo: set workflow status (fail on error-severity findings)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an AgentShield org standard plus a reusable GitHub Actions workflow template intended to enforce security/structure rules for agent configuration files, and updates CI standards to require the new workflow.
Changes:
- Add
standards/agent-standards.mddefining required agent config files and validation rules. - Add
standards/workflows/agent-shield.ymlworkflow template to enforce agent-file existence, references, secret scanning, permission-bypass detection, and SKILL.md frontmatter checks. - Update
standards/ci-standards.mdto list AgentShield as a required workflow and include it in new-repo onboarding.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| standards/workflows/agent-shield.yml | New CI workflow template implementing AgentShield checks. |
| standards/ci-standards.md | Documents AgentShield as the 7th required workflow and updates onboarding checklist. |
| standards/agent-standards.md | New standard describing required agent files and the AgentShield rule set. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The bypass scan now skips lines that are documenting security rules (negation patterns like "No ...", "Must not ...") and backtick-quoted references to avoid false positives in AGENTS.md standards files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@standards/ci-standards.md`:
- Around line 11-14: The compliance script's REQUIRED_WORKFLOWS array in
scripts/compliance-audit.sh still lists only six workflows; add
"agent-shield.yml" to the REQUIRED_WORKFLOWS variable so the script enforces the
new 7-workflow standard (locate the REQUIRED_WORKFLOWS declaration in
scripts/compliance-audit.sh and append "agent-shield.yml" to the array, update
any related counts/validation logic that assumes six workflows).
In `@standards/workflows/agent-shield.yml`:
- Around line 43-48: The workflow step "Validate AGENTS.md references org
standards" is currently emitting a non-blocking ::warning; change it to fail the
job to match the documented "error" severity by emitting ::error and exiting
non-zero when the org reference is missing: update the run block inside the step
to use echo "::error file=AGENTS.md::..." (or both echo plus exiting) and call
exit 1 (or set status=1) when the grep check fails; alternatively, if you prefer
to keep it as a warning, update the documented severity in
standards/agent-standards.md (the org-reference rule) to "warning" so docs and
the step stay consistent.
- Around line 70-77: The for-loop building the file list using command
substitution with $(find ...) causes word-splitting and breaks on filenames with
spaces; replace that loop with a null-delimited safe iteration using find
-print0 piped into while IFS= read -r -d '' file; do ... done (or use find ...
-print0 | xargs -0 -n1 -I{} sh -c '...'), keeping the same inner checks (test -f
"$file", the grep invocation checking
'dangerouslySkipPermissions\|skipPermissions\|--dangerously', the error echo
that references $file, and setting status=1) so you avoid splitting filenames
and preserve the existing logic around file, grep and status.
- Around line 93-99: The frontmatter extraction using sed ('sed -n
'1,/^---$/p'') incorrectly matches the opening delimiter and yields an empty
frontmatter; update the extraction for the frontmatter variable to read the
lines between the first and second '---' delimiters (not including the
delimiters) — replace the current sed invocation that sets frontmatter with a
command that correctly extracts the block (for example use an awk-based
extraction that skips the first '---' and prints until the next '---'), keeping
the same variables (frontmatter and file) and preserving the existing
empty-check logic that emits "::error file=$file::SKILL.md has empty
frontmatter" and sets status=1.
- Around line 55-62: The for-loop over find output will break on filenames with
spaces; replace the "for file in CLAUDE.md AGENTS.md $(find ...)" loop with a
null-delimited safe iteration: use find ... -print0 and a while loop such as
"while IFS= read -r -d '' file; do ... done < <(find ... -print0)" so each
filename (including spaces) is read into $file safely; keep the inner checks (if
[ -f "$file" ]; then and grep -Pn "$patterns" "$file" ...) and preserve setting
status=1 and the error echo.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 497d8435-ed4e-46f5-a7d9-8e9773f20c00
📒 Files selected for processing (3)
standards/agent-standards.mdstandards/ci-standards.mdstandards/workflows/agent-shield.yml
…tter, severity alignment - Add header comment to workflow template - Mask secret values in logs (report count, not content) - Use while-read loops instead of for-$(find) to handle paths with spaces - Expand permission bypass scan to all src/ and .claude/ files - Fix frontmatter extraction to use closing delimiter correctly - Promote org-reference check from warning to error - Mark planned-but-unimplemented rules as "Planned" in standards doc Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@standards/workflows/agent-shield.yml`:
- Around line 143-153: The script computes closing_line relative to tail -n +2
but then uses it as an absolute line number when extracting frontmatter, causing
off-by-one truncation; fix by converting closing_line back to the file's
absolute line number (e.g., add 1) or by extracting frontmatter using the same
relative indexing (use tail/sed consistently) so that frontmatter=$(sed -n
"2,$((closing_line))p" "$file" ) references the correct range; update the logic
around closing_line and the frontmatter extraction in the section that sets
closing_line and frontmatter to ensure the delimiters and content are captured
correctly.
- Around line 131-173: The workflow step "Validate SKILL.md frontmatter"
currently only checks YAML frontmatter (using the find . -name 'SKILL.md' loop
and status variable) but does not enforce the documented no-orphan-skills rule;
add a new non-blocking check after frontmatter validation that collects the list
of skill directories (from the same find of SKILL.md files), reads module names
from module-help.csv, compares them, and emits a warning (echo "::warning
file=...::...") for each SKILL.md whose directory is not listed in
module-help.csv; ensure this check does not set status to 1 so it remains
warning-only and reference the existing find/loop and module-help.csv when
implementing.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 882978cb-b750-468a-848b-c268ad8a4b3a
📒 Files selected for processing (2)
standards/agent-standards.mdstandards/workflows/agent-shield.yml
Replace custom secrets/permission shell scripts with the real AgentShield GitHub Action (v1.4.0) which provides 102 rules across secrets, permissions, hooks, MCP servers, and agent config review. Keep org-specific structural checks (required files, cross-references, SKILL.md frontmatter) as a second layer for petry-projects conventions not covered by the generic scanner. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The action's dist bundle has a missing yaml dependency. The CLI (ecc-agentshield) via npx works correctly and provides the same 102-rule security scan. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The CLI exits non-zero on critical findings automatically (exit code 2). The --fail-on-findings flag only exists in the GitHub Action inputs, not the CLI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
This was referenced Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
standards/agent-standards.mddefines security and structural requirements for agent config files (CLAUDE.md, AGENTS.md, SKILL.md, .claude configs)standards/workflows/agent-shield.ymlis a reusable CI workflow that enforces the standardstandards/ci-standards.mdnow lists AgentShield as the 7th required workflow and includes it in the new-repo onboarding checklistWhy
The org already requires AGENTS.md and CLAUDE.md in every repo (the compliance audit checks for them), but there is no CI workflow that validates these files for security and correctness. Repos with BMAD modules or Claude plugins have agent config files that should be validated for secrets, permission bypasses, and structural integrity.
What the AgentShield workflow checks
Security rules (6)
no-secrets— no API keys, tokens, passwords in agent configsno-skip-permissions— nodangerouslySkipPermissionsbypassesno-unrestricted-tools— tool authorizations should be scopedorg-reference— AGENTS.md must reference org-level standardsclaude-reference— CLAUDE.md must reference AGENTS.mdno-prompt-injection-vectors— no user-controllable template vars in security positionsStructural rules (3)
valid-yaml-frontmatter— SKILL.md files must have valid frontmattermanifest-consistency— skill manifests must match directory structureno-orphan-skills— skills must be registered in module-help.csvTest plan
🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation
Chores