Problem
AgentRC's readiness assessment currently operates at the repository level — it checks for instruction files, lockfiles, CI configs, and other repo-scoped signals. But during customer engagements, field teams also need to assess organization-level and GitHub-settings-level readiness that determines whether AI tooling can be effective across an org's repos.
These signals are queryable via the GitHub API but aren't captured by any AgentRC pillar today. Without them, a repo can score well on readiness while the org-level configuration actually prevents effective Copilot usage.
Signals to assess
GitHub org settings (via API)
- Copilot enablement: Is GitHub Copilot enabled for the org? Which policy (all members, selected teams, disabled)?
- Copilot Chat in IDE: Enabled or restricted?
- Code referencing: Public code filter on or off?
- Content exclusions: Are there Copilot content exclusion patterns that might affect the assessed repos?
Repository-level GitHub settings
- Branch protection rules: Are default branch protections configured? (Required reviews, status checks, signed commits)
- Code scanning / CodeQL: Enabled? Which languages covered?
- Dependabot: Enabled for security alerts? Version updates configured?
- Secret scanning: Enabled with push protection?
- Actions permissions: Which actions are allowed? Are reusable workflows enforced?
Cross-repo patterns
- Template repos: Does the org use template repos that include AI instruction files?
.github repo: Is there a community health .github repo with default instruction files?
- Inner source config: Are there org-level
.github/copilot-instructions.md or instruction packages?
Proposal
New readiness pillar: org-settings
Add an optional pillar that queries org/repo settings when a GitHub token is available:
agentrc readiness --scope=org --org=contoso
Or assess org-level settings for the current repo's org:
agentrc readiness --include-org
Criteria examples
| Criterion ID |
What it checks |
Impact |
copilot-enabled |
Copilot is enabled for the org |
High |
branch-protection |
Default branch has protection rules |
Medium |
code-scanning |
CodeQL or equivalent is configured |
Medium |
dependabot-security |
Dependabot security alerts enabled |
Medium |
secret-scanning |
Secret scanning with push protection |
Medium |
org-instructions |
Org-level .github repo has instruction files |
High |
Auth requirements
This pillar requires a GitHub token with read:org and admin:org scopes. When the token is missing or insufficient, the pillar should gracefully skip with a clear message rather than failing the entire assessment.
Context
- Discussed in "AgentRC: Quick Chat on Futures" meeting
- Distinction made between what's assessable (repo configs, GitHub settings) vs. what's not (cultural practices, team workflows)
- Barton highlighted that org-level signals are critical for customer assessments
- This complements the maturity report card concept — org settings feed into the overall maturity tier
Problem
AgentRC's readiness assessment currently operates at the repository level — it checks for instruction files, lockfiles, CI configs, and other repo-scoped signals. But during customer engagements, field teams also need to assess organization-level and GitHub-settings-level readiness that determines whether AI tooling can be effective across an org's repos.
These signals are queryable via the GitHub API but aren't captured by any AgentRC pillar today. Without them, a repo can score well on readiness while the org-level configuration actually prevents effective Copilot usage.
Signals to assess
GitHub org settings (via API)
Repository-level GitHub settings
Cross-repo patterns
.githubrepo: Is there a community health.githubrepo with default instruction files?.github/copilot-instructions.mdor instruction packages?Proposal
New readiness pillar:
org-settingsAdd an optional pillar that queries org/repo settings when a GitHub token is available:
Or assess org-level settings for the current repo's org:
Criteria examples
copilot-enabledbranch-protectioncode-scanningdependabot-securitysecret-scanningorg-instructions.githubrepo has instruction filesAuth requirements
This pillar requires a GitHub token with
read:organdadmin:orgscopes. When the token is missing or insufficient, the pillar should gracefully skip with a clear message rather than failing the entire assessment.Context