diff --git a/.github/workflows/daily-doc-updater.md b/.github/workflows/daily-doc-updater.md index 9518038..9932257 100644 --- a/.github/workflows/daily-doc-updater.md +++ b/.github/workflows/daily-doc-updater.md @@ -59,11 +59,14 @@ This is **@microsoft/agentrc** — a TypeScript CLI + VS Code extension for prim **Documentation locations:** -- `README.md` — Main product overview, Quick Start, prerequisites, command reference +- `README.md` — Main product overview, Quick Start, prerequisites +- `docs/commands.md` — CLI command reference and options - `CONTRIBUTING.md` — Contribution workflow, code style, release process - `CHANGELOG.md` — Version history -- `docs/product.md` — Product brief, maturity model, architecture decisions -- `docs/plugins.md` — Plugin system documentation, architecture, plugin contract +- `docs/getting-started.md` — Installation, prerequisites, first run walkthrough +- `docs/concepts.md` — Maturity model, readiness pillars, instructions, evaluation +- `docs/dev/product.md` — Product brief, maturity model, architecture decisions +- `docs/dev/plugins.md` — Plugin system documentation, architecture, plugin contract - `.github/copilot-instructions.md` — Copilot coding instructions for the repo - `.github/instructions/*.instructions.md` — Scoped instructions for specific areas - `vscode-extension/README.md` — VS Code extension readme @@ -123,7 +126,7 @@ Key conventions to follow: Review documentation files for: -- New CLI commands or options not yet in README.md +- New CLI commands or options not yet in `docs/commands.md` - New services or APIs not reflected in architecture docs - Changed behavior in existing commands - New VS Code extension features not in the extension README @@ -140,8 +143,10 @@ find vscode-extension -name '*.md' For each missing or incomplete documentation: 1. **Determine the correct file** based on the change type: - - CLI commands/options → `README.md` - - Architecture changes → `docs/product.md` or `docs/plugins.md` + - CLI commands/options → `docs/commands.md` + - Architecture changes → `docs/dev/product.md` or `docs/dev/plugins.md` + - User-facing concepts → `docs/concepts.md` + - Getting started content → `docs/getting-started.md` - Development workflow → `CONTRIBUTING.md` - Copilot coding context → `.github/copilot-instructions.md` - Area-specific instructions → `.github/instructions/*.instructions.md` diff --git a/README.md b/README.md index 2e37a6d..77f9fa7 100644 --- a/README.md +++ b/README.md @@ -1,283 +1,101 @@ # AgentRC -> Prime your repositories for AI-assisted development. +**Context engineering for AI coding agents.** [![CI](https://github.com/microsoft/agentrc/actions/workflows/ci.yml/badge.svg)](https://github.com/microsoft/agentrc/actions/workflows/ci.yml) [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) > [!WARNING] -> **Experimental** — This project is under active development. Expect breaking changes to commands, APIs, and output formats. Ready for early adopter feedback — [open an issue](https://github.com/microsoft/agentrc/issues). +> **Experimental** — Under active development. Expect breaking changes. [Open an issue](https://github.com/microsoft/agentrc/issues) with feedback. -AI coding agents are only as effective as the context they receive. AgentRC is a CLI and VS Code extension that closes the gap — from a single repo to hundreds across your org. +--- -**Measure** — Analyze repo structure and score readiness across a 5-level maturity model. -**Generate** — Produce tailored instructions, evals, and dev configs using the Copilot SDK. -**Maintain** — Run evaluations in CI to catch instruction drift as code evolves. +AI coding agents work best when they know how to build, test, and lint your code — plus your architecture, conventions, and the external services your team relies on. Most repos ship none of that. -![AgentRC — Measure, Generate, Maintain cycle](docs/assets/agentrc-overview.png) - -## Quick Start - -```bash -# Run directly (no install needed) -npx github:microsoft/agentrc readiness -``` - -`npx github:/agentrc ...` installs from the Git repository and runs the package `prepare` script, which builds the CLI before first use. - -Or install locally: +AgentRC reads your codebase and generates the files that close that gap — then evaluates whether they actually help, so the context doesn't go stale as your code evolves. ```bash -git clone https://github.com/microsoft/agentrc.git -cd agentrc && npm install && npm run build && npm link - -# 1. Inspect the repo shape -agentrc analyze - -# 2. Check how AI-ready your repo is -agentrc readiness - -# 3. Generate instructions -agentrc instructions - -# 4. Generate MCP and VS Code configs -agentrc generate mcp -agentrc generate vscode - -# Or do the guided flow interactively -agentrc init +npx github:microsoft/agentrc ``` -## Prerequisites - -| Requirement | Notes | -| --------------------------------- | ---------------------------------------------------------------- | -| **Node.js 20+** | Runtime | -| **GitHub Copilot CLI** | Bundled with the VS Code Copilot Chat extension | -| **Copilot authentication** | Run `copilot` → `/login` | -| **GitHub CLI** _(optional)_ | For batch processing and PRs: `brew install gh && gh auth login` | -| **Azure DevOps PAT** _(optional)_ | Set `AZURE_DEVOPS_PAT` for Azure DevOps workflows | - -## Commands - -### `agentrc analyze` — Inspect Repository Structure +Works as a CLI, as a [VS Code extension](docs/extension.md), and in your [CI/CD pipeline](docs/ci-integration.md) to monitor drift. No config needed — runs on any repo with Node.js 20+. -Detects languages, frameworks, monorepo/workspace structure, and area mappings: - -```bash -agentrc analyze # terminal summary -agentrc analyze --json # machine-readable analysis -agentrc analyze --output analysis.json # save JSON report -agentrc analyze --output analysis.md # save Markdown report -agentrc analyze --output analysis.json --force # overwrite existing report -``` - -### `agentrc readiness` — Run Readiness Report - -Score a repo across 9 pillars grouped into **Repo Health** and **AI Setup**: - -```bash -agentrc readiness # terminal summary -agentrc readiness --visual # GitHub-themed HTML report -agentrc readiness --per-area # include per-area breakdown -agentrc readiness --output readiness.json # save JSON report -agentrc readiness --output readiness.md # save Markdown report -agentrc readiness --output readiness.html # save HTML report -agentrc readiness --policy ./examples/policies/strict.json # apply a custom policy -agentrc readiness --json # machine-readable JSON -agentrc readiness --fail-level 3 # CI gate: exit 1 if below level 3 -``` - -**Maturity levels:** - -| Level | Name | What it means | -| ----- | ------------ | --------------------------------------------------- | -| 1 | Functional | Builds, tests, basic tooling in place | -| 2 | Documented | README, CONTRIBUTING, custom instructions exist | -| 3 | Standardized | CI/CD, security policies, CODEOWNERS, observability | -| 4 | Optimized | MCP servers, custom agents, AI skills configured | -| 5 | Autonomous | Full AI-native development with minimal oversight | - -At Level 2, AgentRC also checks **instruction consistency** — when a repo has multiple AI instruction files (e.g. `copilot-instructions.md`, `CLAUDE.md`, `AGENTS.md`), it detects whether they diverge. Symlinked or identical files pass; diverging files fail with a similarity score and a suggestion to consolidate. - -### `agentrc instructions` — Generate Instructions - -Generate instructions using the Copilot SDK: - -```bash -agentrc instructions # copilot-instructions.md (default) -agentrc instructions --output AGENTS.md # custom output path -agentrc instructions --strategy nested # nested hub + detail files in .agents/ -agentrc instructions --areas # root + all detected areas -agentrc instructions --area frontend # single area -agentrc instructions --areas-only # areas only (skip root) -agentrc instructions --dry-run # preview without writing -agentrc instructions --model claude-sonnet-4.6 -``` - -**Concepts:** - -- **Format**: Output file — `copilot-instructions.md` (default) or `AGENTS.md` (via `--output`) -- **Strategy**: `flat` (single file, default) or `nested` (hub + per-topic detail files) -- **Scope**: `root only` (default), `--areas` (root + areas), `--area ` (single area) - -### `agentrc eval` — Evaluate Instructions - -Measure how instructions improve AI responses with a judge model: +![AgentRC — Measure, Generate, Maintain cycle](docs/assets/agentrc-overview.png) -```bash -agentrc eval --init # scaffold eval config from codebase -agentrc eval agentrc.eval.json # run evaluation -agentrc eval --model gpt-4.1 --judge-model claude-sonnet-4.6 -agentrc eval --fail-level 80 # CI gate: exit 1 if pass rate < 80% -``` +## What it does -### `agentrc generate` — Generate Configs +### Measure -> **Note:** `generate instructions` and `generate agents` are deprecated — use `agentrc instructions` directly. +Score your repo’s AI-readiness across 9 pillars and a 5-level maturity model. Find out what context is missing — from basic linting to MCP server configs. ```bash -agentrc generate mcp # .vscode/mcp.json -agentrc generate vscode --force # .vscode/settings.json (overwrite) +npx github:microsoft/agentrc readiness ``` -### `agentrc batch` / `agentrc pr` — Batch & PRs - -```bash -agentrc batch # interactive TUI (GitHub) -agentrc batch --provider azure # Azure DevOps -agentrc batch owner/repo1 owner/repo2 --json -agentrc batch-readiness --output team.html -agentrc pr owner/repo-name # clone → generate → open PR -``` +### Generate -### `agentrc tui` — Interactive Mode +Produce tailored instruction files, evals, and dev configs via the Copilot SDK. No templates — AgentRC reads your actual code and generates context specific to your stack. ```bash -agentrc tui -``` - -### `agentrc init` — Init Repository - -Interactive or headless repo onboarding — analyzes your stack and generates instructions. For monorepos, auto-detects workspaces and bootstraps `agentrc.config.json` with workspace and area definitions. - -### Global Options - -All commands support `--json` (structured JSON to stdout) and `--quiet` (suppress stderr). JSON output uses a `CommandResult` envelope: - -```json -{ "ok": true, "status": "success", "data": { ... } } +npx github:microsoft/agentrc instructions ``` -### Readiness Policies +### Maintain -Policies customize scoring criteria, override metadata, and tune thresholds: +Context goes stale as your codebase evolves. Evaluate whether your instructions still improve agent responses, and run the check in CI so drift doesn't slip through. ```bash -agentrc readiness --policy ./examples/policies/strict.json -agentrc readiness --policy ./examples/policies/strict.json,./my-overrides.json # chain multiple +npx github:microsoft/agentrc eval ``` -```json -{ - "name": "my-org-policy", - "criteria": { - "disable": ["lint-config"], - "override": { "readme": { "impact": "high", "level": 2 } } - }, - "extras": { "disable": ["pre-commit"] }, - "thresholds": { "passRate": 0.9 } -} -``` +## Works at every scale -Policies can also be set in `agentrc.config.json` (`{ "policies": ["./my-policy.json"] }`). - -### Configuration File - -`agentrc.config.json` (repo root or `.github/`) configures areas, workspaces, and policies: - -```json -{ - "areas": [{ "name": "docs", "applyTo": "docs/**" }], - "workspaces": [ - { - "name": "frontend", - "path": "packages/frontend", - "areas": [ - { "name": "app", "applyTo": "app/**" }, - { "name": "shared", "applyTo": ["shared/**", "common/**"] } - ] - } - ], - "policies": ["./policies/strict.json"] -} -``` +| Workflow | Command | +| ------------------------- | ----------------------------------------- | +| Interactive hub | `agentrc` | +| One-time setup | `agentrc init` | +| CI quality gate | `agentrc readiness --fail-level 3 --json` | +| Batch across an org | `agentrc batch` | +| Automated PR for any repo | `agentrc pr owner/repo` | -- **`areas`** — standalone areas with glob patterns (relative to repo root) -- **`workspaces`** — monorepo sub-projects; each workspace groups areas scoped to a subdirectory. Area `applyTo` patterns are relative to the workspace path. Workspace areas get namespaced names (`frontend/app`) and a `workingDirectory` for scoped eval sessions. -- `agentrc init` auto-detects workspaces (via `.vscode` folders and sibling-area grouping) and bootstraps this file. +Works with **GitHub** and **Azure DevOps**. Supports monorepos, multi-root VS Code workspaces, and custom [policies](docs/policies.md). -> **Security:** Config-sourced policies are restricted to JSON files only — JS/TS module policies must be passed via `--policy`. +## What gets generated -See [docs/plugins.md](docs/plugins.md) for the full plugin authoring guide, including imperative TypeScript plugins, lifecycle hooks, and the trust model. +| File | Purpose | +| --------------------------------- | ------------------------------------------ | +| `.github/copilot-instructions.md` | Teaches AI agents your repo's conventions | +| `.vscode/mcp.json` | Connects AI to your stack's tools and data | +| `.vscode/settings.json` | Tunes VS Code for AI-assisted development | +| `agentrc.eval.json` | Test cases to measure instruction quality | -## Development +> For multi-agent support (Copilot + Claude + others), generate `AGENTS.md` with `--output AGENTS.md`. See [Custom instructions in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-instructions). -```bash -npm run typecheck # type check -npm run lint # ESLint (flat config + Prettier) -npm run test # Vitest tests -npm run test:coverage # with coverage -npm run build # production build via tsup -npx tsx src/index.ts --help # run from source -``` +## Documentation -### VS Code Extension +| | | +| ---------------------------------------------- | ------------------------------------------------------- | +| **[Getting Started](docs/getting-started.md)** | Prerequisites and first run | +| **[Concepts](docs/concepts.md)** | Maturity model, readiness pillars, how generation works | +| **[Commands](docs/commands.md)** | Full CLI reference | +| **[Configuration](docs/configuration.md)** | Areas, workspaces, monorepos | +| **[Policies](docs/policies.md)** | Custom readiness scoring | +| **[At Scale](docs/at-scale.md)** | Batch processing across orgs | +| **[CI Integration](docs/ci-integration.md)** | GitHub Actions & Azure Pipelines | +| **[VS Code Extension](docs/extension.md)** | Sidebar views, commands, settings | +| **[Examples](examples/)** | Configs, evals, and policies | -```bash -cd vscode-extension -npm install && npm run build -# Press F5 to launch Extension Development Host -``` - -See [CONTRIBUTING.md](CONTRIBUTING.md) for workflow and code style guidelines. - -## Project Structure - -``` -packages/core/ -└── src/ - ├── index.ts # Shared public API surface - ├── services/ # Core product logic reused by CLI and extension - │ ├── readiness.ts # 9-pillar scoring engine with pillar groups - │ ├── visualReport.ts # HTML report generator - │ ├── instructions.ts # Copilot SDK integration - │ ├── analyzer.ts # Repo scanning (languages, frameworks, monorepos) - │ ├── evaluator.ts # Eval runner + trajectory viewer - │ ├── generator.ts # MCP/VS Code config generation - │ ├── policy.ts # Readiness policy loading and chain resolution - │ ├── policy/ # Plugin engine (types, compiler, loader, adapter, shadow) - │ ├── git.ts # Git operations (clone, branch, push) - │ ├── github.ts # GitHub API (Octokit) - │ └── azureDevops.ts # Azure DevOps API - └── utils/ # Shared utilities (fs, logger, output) - -src/ -├── cli.ts # Commander CLI wiring -├── commands/ # CLI subcommands (thin orchestrators) -├── index.ts # CLI entrypoint -└── ui/ # Ink/React terminal UI - -vscode-extension/ # VS Code extension shell over packages/core -``` +[Customize AI in VS Code](https://code.visualstudio.com/docs/copilot/customization/overview) · [Custom instructions](https://code.visualstudio.com/docs/copilot/customization/custom-instructions) · [CONTRIBUTING.md](CONTRIBUTING.md) ## Troubleshooting -**"Copilot CLI not found"** — Install the GitHub Copilot Chat extension in VS Code. The CLI is bundled with it. +**"Copilot CLI not found"** — Install the [GitHub Copilot Chat extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) in VS Code. The CLI is bundled with it. + +**"Copilot CLI not logged in"** — Run `copilot` in your terminal, then `/login`. -**"Copilot CLI not logged in"** — Run `copilot` in your terminal, then `/login` to authenticate. +**"GitHub auth required"** — `brew install gh && gh auth login`, or set `GITHUB_TOKEN` (or `GH_TOKEN`). -**"GitHub authentication required"** — Install GitHub CLI (`brew install gh && gh auth login`) or set `GITHUB_TOKEN` / `GH_TOKEN`. +**"Azure DevOps auth required"** — Set `AZURE_DEVOPS_PAT` or `AZDO_PAT`. ## License diff --git a/docs/at-scale.md b/docs/at-scale.md new file mode 100644 index 0000000..8294cbb --- /dev/null +++ b/docs/at-scale.md @@ -0,0 +1,89 @@ +# At Scale + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +Everything that works on one repo also works on hundreds. AgentRC supports batch processing across GitHub organizations and Azure DevOps projects. + +## Batch processing + +Generate instructions and configs for multiple repos in one pass: + +```bash +# Interactive TUI — select repos from your org +npx github:microsoft/agentrc batch + +# Headless — GitHub repos +npx github:microsoft/agentrc batch owner/repo1 owner/repo2 + +# Headless — Azure DevOps repos +npx github:microsoft/agentrc batch org/project/repo1 org/project/repo2 --provider azure +``` + +Repos can also be piped via stdin (one per line): + +```bash +gh repo list my-org --json nameWithOwner -q '.[].nameWithOwner' \ + | npx github:microsoft/agentrc batch +``` + +### Options + +| Flag | Default | Description | +| ----------------- | ------------------- | -------------------- | +| `--output ` | | Write results JSON | +| `--provider

` | `github` | `github` or `azure` | +| `--model ` | `claude-sonnet-4.6` | Model for generation | +| `--branch ` | | Branch name for PRs | + +## Consolidated readiness + +Get a single readiness report across all repos in your org: + +```bash +npx github:microsoft/agentrc batch-readiness --output team.html +``` + +The HTML report shows per-repo scores, trends, and which repos need attention. + +| Flag | Default | Description | +| -------------------- | ------- | -------------------------------------------- | +| `--output ` | | Write HTML report | +| `--policy ` | | Comma-separated policy paths or npm packages | + +## Automated PRs + +Clone a repo, generate configs, and open a PR — all in one command: + +```bash +# GitHub +npx github:microsoft/agentrc pr owner/repo-name + +# Azure DevOps +npx github:microsoft/agentrc pr org/project/repo --provider azure +``` + +| Flag | Default | Description | +| ----------------- | ----------------------- | -------------------- | +| `--branch ` | `agentrc/add-ai-config` | Branch name | +| `--provider

` | | `github` or `azure` | +| `--model ` | `claude-sonnet-4.6` | Model for generation | + +## Authentication + +| Provider | Required env var | +| ------------ | --------------------------------------------------- | +| GitHub | `GITHUB_TOKEN` or `GH_TOKEN`, or `gh` CLI logged in | +| Azure DevOps | `AZURE_DEVOPS_PAT` or `AZDO_PAT` | + +## Repo format + +| Provider | Format | Example | +| ------------ | ------------------ | -------------------------- | +| GitHub | `owner/repo` | `microsoft/agentrc` | +| Azure DevOps | `org/project/repo` | `contoso/platform/backend` | + +## Next steps + +- [Policies](policies.md) — enforce org-specific standards across all repos +- [CI Integration](ci-integration.md) — add readiness gates to every repo's pipeline +- [Commands](commands.md) — full CLI reference for batch, batch-readiness, and pr diff --git a/docs/ci-integration.md b/docs/ci-integration.md new file mode 100644 index 0000000..aefe221 --- /dev/null +++ b/docs/ci-integration.md @@ -0,0 +1,130 @@ +# CI Integration + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +AgentRC commands return structured output and exit codes designed for CI pipelines. + +## Prerequisites + +- **Node.js 20+** on the runner +- **Auth token** — GitHub: `GITHUB_TOKEN` or `GH_TOKEN`. Azure DevOps: `AZURE_DEVOPS_PAT` or `AZDO_PAT`. +- **Copilot CLI** — required for `eval` (it calls the Copilot SDK). Not needed for `readiness`. See the [VS Code Copilot Chat extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) docs for installation. + +> `readiness` is a pure static analysis — it works anywhere Node runs. `eval` invokes AI models via the Copilot SDK, so the runner needs Copilot CLI installed and authenticated. + +## Readiness gate + +Fail if the repo drops below a maturity level: + +```bash +agentrc readiness --fail-level 3 --json +``` + +Exits with code 1 if the readiness level is below 3. The `--json` flag outputs a machine-readable result to stdout. + +## Eval gate + +Fail if instruction quality drops below a pass rate: + +```bash +agentrc eval agentrc.eval.json --fail-level 80 --json +``` + +Exits with code 1 if the pass rate is below 80%. + +## GitHub Actions + +```yaml +name: AgentRC checks +on: [pull_request] + +jobs: + readiness: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 20 + + - name: Check readiness + run: npx github:microsoft/agentrc readiness --fail-level 3 --json + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Run eval + run: npx github:microsoft/agentrc eval --fail-level 80 --json + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # eval requires the GitHub Copilot CLI to be installed + # and authenticated on the runner (copilot → /login) +``` + +## Azure Pipelines + +```yaml +trigger: + - main + +pool: + vmImage: ubuntu-latest + +steps: + - task: NodeTool@0 + inputs: + versionSpec: "20.x" + + - script: npx github:microsoft/agentrc readiness --fail-level 3 --json + displayName: Check readiness + env: + GITHUB_TOKEN: $(GITHUB_TOKEN) # for GitHub-hosted repos + # AZURE_DEVOPS_PAT: $(AZURE_DEVOPS_PAT) # for Azure DevOps repos + + - script: npx github:microsoft/agentrc eval --fail-level 80 --json + displayName: Run eval + env: + GITHUB_TOKEN: $(GITHUB_TOKEN) + # AZURE_DEVOPS_PAT: $(AZURE_DEVOPS_PAT) + # eval requires the GitHub Copilot CLI to be installed + # and authenticated on the runner (copilot → /login) +``` + +## Applying policies in CI + +Use a policy to enforce org-specific standards: + +```bash +npx github:microsoft/agentrc readiness --policy ./policies/strict.json --fail-level 3 --json +``` + +## Any CI system + +The pattern works in any CI that has Node.js: + +```bash +npx github:microsoft/agentrc readiness --fail-level 3 --json +npx github:microsoft/agentrc eval --fail-level 80 --json +``` + +Both commands exit 0 on success and 1 when `--fail-level` is breached. Use `--json` to get structured output for downstream tooling. + +## Output format + +Both commands output a `CommandResult` envelope when `--json` is set: + +```json +{ + "ok": true, + "status": "success", + "data": { ... } +} +``` + +Status values: `"success"`, `"partial"`, `"noop"`, `"error"`. The process exit code is 0 for success and 1 when `--fail-level` is breached or a command error occurs. + +## Next steps + +- [At Scale](at-scale.md) — batch processing and automated PRs across orgs +- [Policies](policies.md) — create org-specific readiness policies +- [Commands](commands.md) — full flag reference for `readiness` and `eval` +- [Concepts](concepts.md) — understand maturity levels and pass rates diff --git a/docs/commands.md b/docs/commands.md new file mode 100644 index 0000000..482da3c --- /dev/null +++ b/docs/commands.md @@ -0,0 +1,215 @@ +# Commands + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +Full CLI reference. Run any command with `npx github:microsoft/agentrc `. All commands support `--json` and `--quiet` global flags. + +## Global options + +| Flag | Effect | +| -------------- | ----------------------------- | +| `--json` | Structured JSON to stdout | +| `--quiet` | Suppress stderr progress | +| `--accessible` | Screen-reader friendly output | + +JSON output uses a `CommandResult` envelope: + +```json +{ "ok": true, "status": "success", "data": { ... } } +``` + +--- + +## `agentrc init` + +Interactive repo onboarding — analyzes your stack and generates instructions and configs in one pass. + +```bash +agentrc init [path] +``` + +| Flag | Default | Description | +| ---------------- | ------------------- | ----------------------------------- | +| `--github` | | Use a GitHub repository | +| `--provider

` | | `github` or `azure` | +| `--yes` | | Accept all defaults (headless mode) | +| `--force` | | Overwrite existing files | +| `--model ` | `claude-sonnet-4.6` | Model for generation | + +For monorepos, auto-detects workspaces and creates `agentrc.config.json`. + +--- + +## `agentrc analyze` + +Inspect repo structure — languages, frameworks, monorepo layout, areas. + +```bash +agentrc analyze [path] +``` + +| Flag | Default | Description | +| ----------------- | ------- | ------------------------------- | +| `--output ` | | Write report (`.json` or `.md`) | +| `--force` | | Overwrite existing output | + +--- + +## `agentrc readiness` + +Score readiness across 9 pillars. See [Concepts — Readiness pillars](concepts.md#readiness-pillars). + +```bash +agentrc readiness [path] +``` + +| Flag | Default | Description | +| -------------------- | ------- | -------------------------------------------- | +| `--output ` | | Write report (`.json`, `.md`, or `.html`) | +| `--force` | | Overwrite existing output | +| `--visual` | | Generate HTML report | +| `--per-area` | | Include per-area breakdown | +| `--policy ` | | Comma-separated policy paths or npm packages | +| `--fail-level ` | | Exit 1 if maturity level < n (1–5) | + +--- + +## `agentrc instructions` + +Generate tailored instruction files via the Copilot SDK. + +```bash +agentrc instructions +``` + +| Flag | Default | Description | +| ------------------- | --------------------------------- | --------------------------------------- | +| `--repo ` | cwd | Repository path | +| `--output ` | `.github/copilot-instructions.md` | Output file | +| `--model ` | `claude-sonnet-4.6` | Model for generation | +| `--force` | | Overwrite existing area files | +| `--strategy ` | `flat` | `flat` or `nested` | +| `--areas` | | Also generate for detected areas | +| `--areas-only` | | Only area instructions (skip root) | +| `--area ` | | Single area | +| `--claude-md` | | Also generate `CLAUDE.md` (nested only) | +| `--dry-run` | | Preview without writing | + +See [Concepts — Instructions](concepts.md#instructions) for strategy and area details. + +--- + +## `agentrc eval` + +Evaluate instruction quality with a judge model. + +```bash +agentrc eval [path] # path to eval config (default: ./agentrc.eval.json) +agentrc eval --init # scaffold test cases +``` + +| Flag | Default | Description | +| ---------------------- | ------------------- | ---------------------------------------------- | +| `--repo ` | cwd | Repository path | +| `--model ` | `claude-sonnet-4.6` | Model for responses | +| `--judge-model ` | `claude-sonnet-4.6` | Model for judging | +| `--list-models` | | List available models and exit | +| `--output ` | | Write results JSON | +| `--init` | | Scaffold starter eval config | +| `--count ` | `5` | Number of test cases to generate with `--init` | +| `--fail-level ` | | Exit 1 if pass rate < n% | + +See [Concepts — Evaluation](concepts.md#evaluation) for how scoring works. + +--- + +## `agentrc generate` + +Generate MCP and VS Code configs. + +```bash +agentrc generate [path] +``` + +Types: `mcp`, `vscode` (also `instructions` and `agents`, but deprecated — use `agentrc instructions`). + +| Flag | Default | Description | +| ------------------- | ------------------- | ------------------------ | +| `--force` | | Overwrite existing files | +| `--model ` | `claude-sonnet-4.6` | Model for generation | +| `--strategy ` | `flat` | Instruction strategy | + +--- + +## `agentrc batch` + +Batch-process repos across a GitHub org or Azure DevOps project. See [At Scale](at-scale.md) for detailed usage and examples. + +```bash +agentrc batch # interactive TUI +agentrc batch owner/repo1 owner/repo2 # GitHub (headless) +agentrc batch org/project/repo --provider azure # Azure DevOps +``` + +| Flag | Default | Description | +| ----------------- | ------------------- | -------------------- | +| `--output ` | | Write results JSON | +| `--provider

` | `github` | `github` or `azure` | +| `--model ` | `claude-sonnet-4.6` | Model for generation | +| `--branch ` | | Branch name for PRs | + +Repos can also be piped via stdin (one per line). + +--- + +## `agentrc batch-readiness` + +Consolidated readiness report across multiple repos. + +```bash +agentrc batch-readiness --output team.html +``` + +| Flag | Default | Description | +| -------------------- | ------- | -------------------------------------------- | +| `--output ` | | Write HTML report | +| `--policy ` | | Comma-separated policy paths or npm packages | + +--- + +## `agentrc pr` + +Clone a repo, generate configs, and open a PR. See [At Scale](at-scale.md) for the full workflow. + +```bash +agentrc pr owner/repo-name # GitHub +agentrc pr org/project/repo --provider azure # Azure DevOps +``` + +| Flag | Default | Description | +| ----------------- | ----------------------- | -------------------- | +| `--branch ` | `agentrc/add-ai-config` | Branch name | +| `--provider

` | | `github` or `azure` | +| `--model ` | `claude-sonnet-4.6` | Model for generation | + +--- + +## `agentrc tui` + +Interactive terminal UI for all workflows. + +```bash +agentrc tui +``` + +| Flag | Default | Description | +| ---------------- | ------- | -------------------- | +| `--repo ` | cwd | Repository path | +| `--no-animation` | | Skip animated banner | + +## Next steps + +- [Concepts](concepts.md) — understand maturity model and readiness pillars +- [Configuration](configuration.md) — `agentrc.config.json` for areas, workspaces, monorepos +- [At Scale](at-scale.md) — batch processing and automated PRs +- [CI Integration](ci-integration.md) — use `--fail-level` and `--json` in pipelines diff --git a/docs/concepts.md b/docs/concepts.md new file mode 100644 index 0000000..7cda865 --- /dev/null +++ b/docs/concepts.md @@ -0,0 +1,197 @@ +# Concepts + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +AgentRC is built around a simple loop: **measure** how AI-ready a repo is, **generate** the files that close the gaps, and **maintain** quality as code evolves. This page explains the key ideas behind each step. + +## The AI-readiness problem + +AI coding agents are only as effective as the context they receive. A Copilot chat session that knows your repo uses React 19 with Server Components, follows a specific naming convention, and has a particular testing strategy will produce dramatically better code than one that doesn't. + +Most repos lack this structured context. AgentRC fills the gap by analyzing your repo and generating the instruction files, configs, and evaluations that teach AI agents how your codebase works. + +## Maturity model + +AgentRC maps every repo to a 5-level maturity model: + +| Level | Name | What it means | +| ----- | ---------------- | --------------------------------------------------- | +| 1 | **Functional** | Builds, tests, basic tooling in place | +| 2 | **Documented** | README, CONTRIBUTING, custom instructions exist | +| 3 | **Standardized** | CI/CD, security policies, CODEOWNERS, observability | +| 4 | **Optimized** | MCP servers, custom agents, AI skills configured | +| 5 | **Autonomous** | Full AI-native development with minimal oversight | + +The maturity level is computed from the readiness score (see below). Use `--fail-level` in CI to enforce a minimum level: + +```bash +agentrc readiness --fail-level 3 +``` + +## Readiness pillars + +The readiness score is based on 9 pillars, grouped into two categories: + +### Repo Health + +These pillars measure general engineering maturity — things that benefit any development workflow, not just AI: + +| Pillar | What it checks | +| ------------------- | ----------------------------------------------------------------------------- | +| **Style** | Linter config (ESLint/Biome/Prettier), type-checking config (TypeScript/Mypy) | +| **Build** | Build script in package.json, CI workflow config | +| **Testing** | Test script in package.json, area-scoped test scripts | +| **Docs** | README, CONTRIBUTING guide, area-scoped READMEs | +| **Dev Environment** | Lockfile (npm/pnpm/yarn/bun), `.env.example` | +| **Code Quality** | Formatter config (Prettier/Biome) | +| **Observability** | Observability dependencies (OpenTelemetry, Pino, Winston, Bunyan) | +| **Security** | LICENSE, CODEOWNERS, SECURITY.md, Dependabot config | + +### AI Setup + +These pillars measure how well the repo is prepared for AI-assisted development: + +| Pillar | What it checks | +| -------------- | ---------------------------------------------------------- | +| **AI Tooling** | Custom instructions, MCP servers, agent configs, AI skills | + +At Level 2+, AgentRC also checks **instruction consistency** — if your repo has multiple instruction files (`copilot-instructions.md`, `AGENTS.md`, `CLAUDE.md`), it detects whether they diverge and suggests consolidation. + +## Instructions + +Instructions are the core output of AgentRC — Markdown files that teach AI coding agents about your repo's conventions, architecture, and preferences. + +### How instructions work in VS Code + +VS Code supports several types of instruction files. AgentRC generates the most common ones: + +| File | Scope | When to use | +| --------------------------------- | -------------------------- | -------------------------------------------------------------- | +| `AGENTS.md` | Always-on, whole workspace | **Recommended** — works with Copilot, Claude, and other agents | +| `.github/copilot-instructions.md` | Always-on, whole workspace | Copilot-only repos | +| `.instructions.md` files | File-pattern or task-based | Targeted rules for specific file types, languages, or folders | + +> **Deep dive:** See [Custom instructions in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-instructions) for the full reference on instruction file types, priority rules, and the YAML frontmatter format. + +### How AgentRC generates instructions + +Instructions are **generated, not templated**. AgentRC uses the Copilot SDK to analyze your actual repo content — languages, frameworks, directory structure, existing conventions — and produce instructions tailored to what it finds. No generic boilerplate. + +```bash +# Generate an AGENTS.md (recommended) +agentrc instructions --output AGENTS.md + +# Generate Copilot-only instructions +agentrc instructions +``` + +### Flat vs nested strategy + +For larger repos, you can choose how instructions are organized: + +- **`flat`** (default) — A single instructions file at the repo root. Simple, easy to review. +- **`nested`** — A hub file at the root plus per-topic detail files in an `.agents/` directory. Better for large repos where a single file would be unwieldy. + +```bash +agentrc instructions --strategy nested +``` + +### Area-scoped instructions + +In monorepos, different parts of the codebase often need different instructions. AgentRC supports generating per-area instructions: + +```bash +# Generate root + all detected areas +agentrc instructions --areas + +# Generate for a single area +agentrc instructions --area frontend + +# Generate areas only (skip root) +agentrc instructions --areas-only +``` + +Areas are defined in `agentrc.config.json` or auto-detected by `agentrc init`. See [Configuration](configuration.md) for details. + +> **Tip:** Per-area instructions map well to VS Code's `.instructions.md` files with `applyTo` patterns. See [File-based instructions in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-instructions#_use-instructionsmd-files) for how VS Code discovers and applies them. + +## Evaluation + +How do you know your instructions actually help? AgentRC includes an evaluation framework that measures the impact of instructions on AI responses. + +### How it works + +1. **Scaffold test cases** from your codebase — AgentRC generates scenarios that test whether the AI follows your repo's conventions: + + ```bash + agentrc eval --init + ``` + +2. **Run the evaluation** — For each test case, the evaluator compares AI responses with and without your instructions: + + ```bash + agentrc eval agentrc.eval.json + ``` + +3. **Score with a judge model** — A separate AI model evaluates whether the instructed response is better, producing a pass/fail verdict per test case. + +### Using evaluation as a CI gate + +Set a minimum pass rate to prevent instruction drift: + +```bash +agentrc eval agentrc.eval.json --fail-level 80 +``` + +This exits with code 1 if the pass rate drops below 80%, making it easy to integrate into GitHub Actions or Azure Pipelines. + +## Configs + +Beyond instructions, AgentRC generates development environment configs: + +- **`.vscode/mcp.json`** — [MCP server](https://code.visualstudio.com/docs/copilot/customization/mcp-servers) configuration based on your detected stack (databases, APIs, frameworks) +- **`.vscode/settings.json`** — VS Code settings tuned for AI-assisted development + +```bash +agentrc generate mcp +agentrc generate vscode +``` + +> **Deep dive:** See [Customize AI in VS Code](https://code.visualstudio.com/docs/copilot/customization/overview) for the full picture of how instructions, MCP servers, prompt files, custom agents, and other customizations work together. + +## Policies + +Policies let organizations customize the readiness scoring to match their standards. A policy can: + +- **Disable** criteria that don't apply to your stack +- **Override** scoring metadata (impact, required level) +- **Set thresholds** for pass rates + +```bash +agentrc readiness --policy ./policies/strict.json +``` + +AgentRC ships with example policies for common scenarios: + +| Policy | Use case | +| ----------------------- | -------------------------------------------------- | +| `strict.json` | High bar — all criteria enabled, strict thresholds | +| `ai-only.json` | Focus on AI setup pillars only | +| `repo-health-only.json` | Focus on repo health pillars only | + +See [Policies](policies.md) for the full policy authoring guide. + +## Batch workflows + +Everything that works on one repo also works on hundreds. See [At Scale](at-scale.md) for batch processing across GitHub organizations and Azure DevOps projects. + +## What's next + +- [Getting Started](getting-started.md) — install and run your first assessment +- [Commands](commands.md) — full CLI reference +- [Configuration](configuration.md) — set up `agentrc.config.json` for monorepos +- [Policies](policies.md) — author custom readiness policies +- [At Scale](at-scale.md) — batch processing and automated PRs across orgs +- [CI Integration](ci-integration.md) — GitHub Actions and Azure Pipelines +- [VS Code Extension](extension.md) — sidebar views, commands, settings +- [Customize AI in VS Code](https://code.visualstudio.com/docs/copilot/customization/overview) — the full customization ecosystem diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..96ca4b2 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,164 @@ +# Configuration + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +AgentRC uses `agentrc.config.json` to define areas, workspaces, and policies for your repo. This file is optional for single-project repos — `agentrc init` creates it automatically when it detects a monorepo. + +## File location + +Place `agentrc.config.json` at the repo root or in `.github/`. AgentRC checks both. + +## Schema + +```json +{ + "areas": [], + "workspaces": [], + "policies": [], + "strategy": "flat", + "detailDir": ".agents", + "claudeMd": false +} +``` + +All fields are optional. + +## Areas + +Areas are named slices of your codebase that get their own instructions. Each area has a glob pattern that scopes it to specific files: + +```json +{ + "areas": [ + { "name": "docs", "applyTo": "docs/**" }, + { "name": "api", "applyTo": "src/api/**", "description": "REST API layer" }, + { "name": "validation", "applyTo": ["src/validation/**", "src/schemas/**"] } + ] +} +``` + +| Field | Type | Required | Description | +| ------------- | ------------------ | -------- | ----------------------------------------- | +| `name` | string | yes | Unique area identifier | +| `applyTo` | string or string[] | yes | Glob pattern(s) relative to repo root | +| `description` | string | no | Human description (used in generation) | +| `parentArea` | string | no | Parent area name for hierarchical nesting | + +Generate area instructions with: + +```bash +agentrc instructions --areas # root + all areas +agentrc instructions --area docs # single area +agentrc instructions --areas-only # areas only, skip root +``` + +Per-area instructions map to VS Code's `.instructions.md` files with `applyTo` frontmatter. See [File-based instructions in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-instructions#_use-instructionsmd-files). + +## Workspaces + +Workspaces model monorepo sub-projects. Each workspace groups areas scoped to a subdirectory: + +```json +{ + "workspaces": [ + { + "name": "frontend", + "path": "packages/frontend", + "areas": [ + { "name": "app", "applyTo": "app/**" }, + { "name": "shared", "applyTo": ["shared/**", "common/**"] } + ] + }, + { + "name": "backend", + "path": "packages/backend", + "areas": [ + { "name": "api", "applyTo": "src/api/**" }, + { "name": "workers", "applyTo": "src/workers/**", "parentArea": "api" } + ] + } + ] +} +``` + +| Field | Type | Required | Description | +| ------- | ------ | -------- | ------------------------------ | +| `name` | string | yes | Workspace identifier | +| `path` | string | yes | Relative path from repo root | +| `areas` | Area[] | yes | Areas scoped to this workspace | + +Key behaviors: + +- Area `applyTo` patterns are relative to the workspace `path`, not the repo root +- Workspace areas get namespaced names: `frontend/app`, `backend/api` +- Each workspace area gets its own `workingDirectory` for scoped eval sessions + +## Instruction strategy + +Control how generated instructions are organized: + +```json +{ + "strategy": "nested", + "detailDir": ".agents", + "claudeMd": true +} +``` + +| Field | Default | Description | +| ----------- | ----------- | ------------------------------------------------------------- | +| `strategy` | `"flat"` | `"flat"` = single file, `"nested"` = hub + detail files | +| `detailDir` | `".agents"` | Folder for detail files in nested strategy | +| `claudeMd` | `false` | Also generate `CLAUDE.md` alongside `AGENTS.md` (nested only) | + +CLI flags override config values: `--strategy nested` takes precedence. + +## Policies + +Reference policy files to customize readiness scoring: + +```json +{ + "policies": ["./policies/strict.json"] +} +``` + +Paths are resolved relative to the current working directory. Only JSON policy files are allowed in config — TypeScript module policies must be passed via `--policy` on the command line. + +See [Policies](policies.md) for the policy authoring guide. + +## Full example + +```json +{ + "areas": [{ "name": "docs", "applyTo": "docs/**", "description": "Documentation" }], + "workspaces": [ + { + "name": "frontend", + "path": "packages/frontend", + "areas": [ + { "name": "app", "applyTo": "app/**" }, + { "name": "shared", "applyTo": ["shared/**", "common/**"] } + ] + }, + { + "name": "backend", + "path": "packages/backend", + "areas": [ + { "name": "api", "applyTo": "src/api/**" }, + { "name": "workers", "applyTo": "src/workers/**", "parentArea": "api" } + ] + } + ], + "policies": ["./policies/strict.json"], + "strategy": "flat" +} +``` + +See [examples/agentrc.config.json](../examples/agentrc.config.json) for a working example. + +## Next steps + +- [Policies](policies.md) — customize readiness scoring +- [Commands](commands.md) — CLI flags that override config values +- [Concepts](concepts.md) — how areas and workspaces fit the maturity model diff --git a/docs/plugins.md b/docs/dev/plugins.md similarity index 100% rename from docs/plugins.md rename to docs/dev/plugins.md diff --git a/docs/product.md b/docs/dev/product.md similarity index 100% rename from docs/product.md rename to docs/dev/product.md diff --git a/docs/extension.md b/docs/extension.md new file mode 100644 index 0000000..25f7f4e --- /dev/null +++ b/docs/extension.md @@ -0,0 +1,71 @@ +# VS Code Extension + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +The AgentRC extension brings all CLI capabilities into VS Code with tree views, visual reports, and command palette access. + +## Install + +The extension is not yet published to the Marketplace. Build from source: + +```bash +cd vscode-extension && npm install && node esbuild.mjs +``` + +Then press `F5` in VS Code to launch the Extension Development Host. + +## Commands + +Open the Command Palette (`⇧⌘P`) and type `AgentRC`: + +| Command | What it does | +| ------------------------------------- | ------------------------------------------- | +| **Init Repository** | Guided onboarding (analyze → generate) | +| **Analyze Repository** | Detect languages, frameworks, areas | +| **Readiness Report** | Score and display readiness | +| **Generate Instructions** | Generate instruction files via Copilot SDK | +| **Generate Configs** | Generate MCP and VS Code configs | +| **Run Eval** | Evaluate instruction quality | +| **Scaffold Eval Config** | Create starter `agentrc.eval.json` | +| **Create Pull Request** | Clone → generate → open PR | +| **Generate Instructions (All Roots)** | Batch generate across multi-root workspaces | + +## Sidebar views + +The AgentRC activity bar icon gives you two tree views: + +- **Analysis** — Repo structure at a glance: languages, frameworks, detected areas +- **Readiness** — Pillar-by-pillar readiness breakdown with pass/fail indicators + +Both views refresh automatically when you run commands. + +## Settings + +Configure via **Settings** (`⌘,`) → search `agentrc`: + +| Setting | Default | Description | +| --------------------- | ------------------- | ------------------------------ | +| `agentrc.model` | `claude-sonnet-4.6` | Default model for generation | +| `agentrc.judgeModel` | _(uses model)_ | Model for eval judging | +| `agentrc.autoAnalyze` | `false` | Auto-analyze on workspace open | + +## Extension vs CLI + +Both use the same core services. Choose based on workflow: + +| Use the extension when... | Use the CLI when... | +| ---------------------------------------- | ---------------------------------- | +| You want visual readiness reports | You need CI/CD integration | +| You prefer command palette over terminal | You're batch-processing repos | +| You want sidebar tree views | You want JSON output for scripting | +| You're working in a single repo | You're automating across an org | + +## Multi-root workspaces + +In multi-root workspaces, commands that target a single repo will prompt you to pick a folder. The **Generate Instructions (All Roots)** command processes all workspace folders at once. + +## Next steps + +- [Getting Started](getting-started.md) — CLI quickstart +- [Commands](commands.md) — full CLI reference (same capabilities, different interface) +- [Configuration](configuration.md) — `agentrc.config.json` for monorepos diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..d24f1ff --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,67 @@ +# Getting Started + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +## Prerequisites + +- **Node.js 20+** +- **GitHub Copilot CLI** — bundled with the [VS Code Copilot Chat extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat). Run `copilot` → `/login` to authenticate. +- **Git host auth** — GitHub: `gh` CLI or `GITHUB_TOKEN`/`GH_TOKEN` env var. Azure DevOps: `AZURE_DEVOPS_PAT` (or `AZDO_PAT`) env var. + +## Run + +No install needed: + +```bash +npx github:microsoft/agentrc +``` + +The TUI is an interactive menu where you can generate instructions, score readiness, run evals, and batch-process repos — all from one place. On first run, start with **init** to scaffold everything in one pass: + +```bash +npx github:microsoft/agentrc init +``` + +`init` analyzes your repo, scores readiness, and generates tailored instructions and configs. Commit the output: + +| File | What it does | +| --------------------------------- | ------------------------------------------ | +| `.github/copilot-instructions.md` | Teaches AI agents your repo’s conventions | +| `.vscode/mcp.json` | Connects AI to your stack's tools and data | +| `.vscode/settings.json` | Tunes VS Code for AI-assisted dev | + +> **Tip:** For multi-agent support (Copilot + Claude + others), also generate `AGENTS.md`: +> +> ```bash +> npx github:microsoft/agentrc instructions --output AGENTS.md +> ``` +> +> See [Custom instructions in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-instructions) for details on instruction file types. + +## Verify + +Measure whether your instructions actually improve AI responses: + +```bash +npx github:microsoft/agentrc eval --init # scaffold test cases +npx github:microsoft/agentrc eval # run evaluation +``` + +## Keep it green + +Add to CI to catch drift: + +```bash +npx github:microsoft/agentrc readiness --fail-level 3 --json # gate on maturity level +npx github:microsoft/agentrc eval --fail-level 80 --json # gate on eval pass rate +``` + +See [CI Integration](ci-integration.md) for full GitHub Actions and Azure Pipelines examples. + +## Next steps + +- [Concepts](concepts.md) — maturity model, readiness pillars, how instructions are generated +- [Commands](commands.md) — full CLI reference +- [Configuration](configuration.md) — `agentrc.config.json` for monorepos +- [At Scale](at-scale.md) — batch processing across GitHub orgs and Azure DevOps +- [Customize AI in VS Code](https://code.visualstudio.com/docs/copilot/customization/overview) — instructions, MCP servers, prompt files, custom agents diff --git a/docs/policies.md b/docs/policies.md new file mode 100644 index 0000000..c117999 --- /dev/null +++ b/docs/policies.md @@ -0,0 +1,160 @@ +# Policies + +> [AgentRC](https://github.com/microsoft/agentrc) — prime your repositories for AI-assisted development. + +Policies customize how AgentRC scores readiness. Use them to disable irrelevant criteria, raise the bar on things that matter, or add custom checks. + +## Using a policy + +```bash +agentrc readiness --policy ./policies/strict.json +agentrc readiness --policy ./base.json,./overrides.json # chain multiple +agentrc readiness --policy @org/agentrc-policy-strict # npm package +``` + +Or set it in [configuration](configuration.md): + +```json +{ + "policies": ["./policies/strict.json"] +} +``` + +## Built-in examples + +AgentRC ships with three example policies in `examples/policies/`: + +| Policy | What it does | +| ----------------------- | ------------------------------------------------------ | +| `strict.json` | 100% pass rate, raises impact on key criteria | +| `ai-only.json` | Disables all repo-health checks, focuses on AI tooling | +| `repo-health-only.json` | Disables AI checks, focuses on traditional quality | + +## Writing a policy + +A policy is a JSON file with three optional sections: + +```json +{ + "name": "my-policy", + "criteria": { ... }, + "extras": { ... }, + "thresholds": { ... } +} +``` + +### Disable criteria + +Remove checks that don't apply to your stack: + +```json +{ + "name": "no-infra-checks", + "criteria": { + "disable": ["env-example", "observability", "dependabot"] + } +} +``` + +### Override criteria + +Change impact, level, or title of existing checks: + +```json +{ + "name": "custom-weights", + "criteria": { + "override": { + "readme": { "impact": "high", "level": 2 }, + "lint-config": { "title": "Linter required" } + } + } +} +``` + +### Set thresholds + +Control the overall pass rate: + +```json +{ + "name": "strict-pass-rate", + "thresholds": { + "passRate": 0.9 + } +} +``` + +### Add custom criteria + +JSON policies can only customize existing criteria (disable, override, set thresholds). To **add new criteria** with custom detection logic, use a TypeScript/JavaScript policy module passed via `--policy`: + +```bash +agentrc readiness --policy ./my-plugin.js +``` + +> `.ts` policy files require a TypeScript-capable runtime (e.g., `npx tsx`) or must be compiled to `.js` first. + +See [Plugin System](dev/plugins.md) for the full plugin API. + +## Chaining policies + +When multiple policies are applied, they merge in order. Later policies override earlier ones. Useful for layering an org baseline with team-specific customizations: + +```bash +agentrc readiness --policy ./org-baseline.json,./team-frontend.json +``` + +## Advanced: TypeScript plugins + +For full control over the 5-stage detection/recommendation pipeline, write a TypeScript plugin module. This is an advanced escape hatch — most use cases are covered by JSON policies. + +See [Plugin System](dev/plugins.md) for the full plugin architecture, lifecycle hooks, and TypeScript API. + +## Scoring reference + +Each recommendation has an impact level that maps to a weight: + +| Impact | Weight | +| -------- | ------ | +| critical | 5 | +| high | 4 | +| medium | 3 | +| low | 2 | +| info | 0 | + +Score = 1 - (total deductions / max possible weight). Grades: A >= 0.9, B >= 0.8, C >= 0.7, D >= 0.6, F < 0.6. + +## Extras + +Extras are lightweight, optional checks for repository best practices. Unlike criteria, extras **never affect the readiness score or pass rate** — they’re reported in a separate section of readiness reports. + +AgentRC ships with four built-in extras: + +| ID | What it checks | +| ------------------ | ----------------------------------------- | +| `agents-doc` | `AGENTS.md` is present | +| `pr-template` | Pull request template exists | +| `pre-commit` | Pre-commit hooks configured (Husky, etc.) | +| `architecture-doc` | Architecture documentation present | + +### Disable extras + +Remove extras that don’t apply: + +```json +{ + "name": "skip-extras", + "extras": { + "disable": ["agents-doc", "pre-commit"] + } +} +``` + +Adding new extras requires a TypeScript plugin — they need a detection function that JSON can’t express. See [Plugin System](dev/plugins.md). + +## Next steps + +- [Configuration](configuration.md) — reference policies from `agentrc.config.json` +- [CI Integration](ci-integration.md) — enforce policies in GitHub Actions and Azure Pipelines +- [Plugin System](dev/plugins.md) — advanced TypeScript plugin architecture