diff --git a/.github/aw/create-agentic-workflow.md b/.github/aw/create-agentic-workflow.md index 0e7257ecab2..7e7705c6996 100644 --- a/.github/aw/create-agentic-workflow.md +++ b/.github/aw/create-agentic-workflow.md @@ -136,6 +136,16 @@ Agentic workflows execute as **a single GitHub Actions job** with the AI agent r - Example: "Run migrations, rollback if deployment fails" - **Alternative**: Use traditional GitHub Actions with conditional steps and job failure handling +### Steering Pattern + +For complex scenarios, use a hybrid model: **agentic control-plane decisions** with a **deterministic workflow backbone**: + +- Agentic layer steers: selects next wave, decides retry scope, and dispatches downstream workflows +- Deterministic jobs execute: approvals, fan-out, builds/tests, and auditable state transitions +- Use workflow outputs/artifacts/tracker IDs for state handoff between runs + +This keeps AI decision-making while preserving reliability for long-running orchestration. + ### How to Handle These Requests When a user requests capabilities beyond agentic workflows: @@ -272,6 +282,63 @@ These resources contain workflow patterns, best practices, safe outputs, and per - **Feature synchronization**: Main repo propagates changes to sub-repos via PRs - **Organization-wide coordination**: Single workflow creates issues across multiple repos + **CentralRepoOps (private control-plane variant):** + - Use when the user wants **one private repository** to coordinate rollout across many repositories + - Keywords: "single private repo", "control repo", "rollout to 100s of repos", "fleet", "governance" + - Prefer **agentic-first explicit notation** in examples and generated markdown: + - Show prompt body first, but keep tunable frontmatter defaults visible in the same file + - Avoid imports for commonly tuned controls (concurrency, rate-limit, tracker-id, safe-outputs) + - Omit `default:` fields when using standard defaults; apply defaults via fallback expressions where values are consumed + - Preferred example style: + + ```aw + --- + on: + workflow_dispatch: + inputs: + objective: { required: true } + rollout_profile: { required: false, default: "standard" } + concurrency: + group: "centralrepoops-${{ github.workflow }}-${{ inputs.rollout_profile }}" + cancel-in-progress: false + rate-limit: + max: 2 + window: 60 + --- + + # + + ## Objective + {{inputs.objective}} + + ## Instructions + Focus on the rollout objective and produce the smallest safe change. + ``` + + - Expansion rule: + - Start with explicit tunable defaults in one file + - Extract only truly fixed internals if the user asks for further simplification + - Always include a short "Tuning" section in generated workflow text: + - Call out exactly which knobs users are expected to change + - Mark deterministic/runtime wiring as "usually keep as-is" + - Prefer profile-based scaffolding for this variant: + - **Lean profile (default):** compact frontmatter + instruction-first prompt + - **Advanced profile (only when requested):** campaign tracking and wave-promotion controls + - Then customize only objective/targets/caps instead of writing from scratch + - Selection rule (minimize choices): + 1. Single-repo demo → simple profile + 2. Multi-repo control plane (most cases) → CentralRepoOps lean profile + 3. Campaign + promotion steering required → CentralRepoOps advanced profile + - Prefer markdown-first authoring (`.github/workflows/.md`) with deterministic jobs + prompt instructions + - Prefer built-in frontmatter controls first: `manual-approval`, `concurrency`, `rate-limit`, `tracker-id` + - Recommended model: + 1. agentic control-plane decisions for steering (next wave, retry scope, mutation intent) + 2. deterministic workflow backbone (fan-out, shard/build validation, approvals, auditable state transitions) + - For campaign-style rollouts, include explicit control-plane metadata as workflow inputs (for example: `campaign_id`, `campaign_goal`) and propagate them into summary/dispatch outputs + - For wave steering, include promotion controls in workflow inputs (for example: `promote_to_next_wave`, `next_rollout_profile`) and publish structured payload via `safe-outputs.dispatch-workflow` + - Prefer a **single starter workflow** for initial adoption, then split into retry/replay workflows as scale grows + - Reference docs: https://github.github.com/gh-aw/patterns/centralrepoops/ + **Architectural Constraints:** - ✅ **CAN**: Create issues/PRs/comments in external repos using `target-repo` - ✅ **CAN**: Read from external repos using GitHub toolsets (repos, issues, actions) diff --git a/.github/workflows/reusable-central-deterministic-gate.yml b/.github/workflows/reusable-central-deterministic-gate.yml new file mode 100644 index 00000000000..cdf39957a79 --- /dev/null +++ b/.github/workflows/reusable-central-deterministic-gate.yml @@ -0,0 +1,198 @@ +name: Reusable Central Deterministic Gate + +on: + workflow_call: + inputs: + rollout_profile: + description: Rollout profile (pilot, standard, broad) + required: false + default: standard + type: string + targets_pilot_json: + description: JSON array of owner/repo targets for pilot profile + required: false + default: "[]" + type: string + targets_standard_json: + description: JSON array of owner/repo targets for standard profile + required: false + default: "[]" + type: string + targets_broad_json: + description: JSON array of owner/repo targets for broad profile + required: false + default: "[]" + type: string + target_ref: + description: Default target git ref for deterministic checkout/build + required: false + default: main + type: string + shard_count: + description: Number of rollout shards + required: false + default: "1" + type: string + shard_index: + description: Zero-based shard index to execute + required: false + default: "0" + type: string + outputs: + primary_target_repo: + description: Primary repository used by the agentic wrapper checkout step + value: ${{ jobs.select_shard.outputs.primary_target_repo }} + target_ref: + description: Target git ref applied to deterministic checkout/build + value: ${{ jobs.resolve_policy.outputs.target_ref }} + +permissions: + contents: read + +jobs: + resolve_policy: + runs-on: ubuntu-latest + outputs: + targets_json: ${{ steps.resolve.outputs.targets_json }} + target_ref: ${{ steps.resolve.outputs.target_ref }} + shard_count: ${{ steps.resolve.outputs.shard_count }} + shard_index: ${{ steps.resolve.outputs.shard_index }} + steps: + - id: resolve + uses: actions/github-script@v7 + env: + ROLLOUT_PROFILE: ${{ inputs.rollout_profile }} + TARGETS_PILOT_JSON: ${{ inputs.targets_pilot_json }} + TARGETS_STANDARD_JSON: ${{ inputs.targets_standard_json }} + TARGETS_BROAD_JSON: ${{ inputs.targets_broad_json }} + TARGET_REF: ${{ inputs.target_ref }} + SHARD_COUNT: ${{ inputs.shard_count }} + SHARD_INDEX: ${{ inputs.shard_index }} + CURRENT_REPOSITORY: ${{ github.repository }} + with: + script: | + const profile = (process.env.ROLLOUT_PROFILE || 'standard').toLowerCase(); + const targetsByProfile = { + pilot: process.env.TARGETS_PILOT_JSON || '[]', + standard: process.env.TARGETS_STANDARD_JSON || '[]', + broad: process.env.TARGETS_BROAD_JSON || '[]', + }; + + if (!Object.prototype.hasOwnProperty.call(targetsByProfile, profile)) { + core.setFailed(`Invalid rollout_profile '${profile}'. Expected one of: pilot, standard, broad`); + return; + } + + let targets = []; + try { + const parsed = JSON.parse(targetsByProfile[profile]); + if (!Array.isArray(parsed) || !parsed.every((repo) => typeof repo === 'string' && repo.includes('/'))) { + core.setFailed(`targets_${profile}_json must be a JSON array of owner/repo strings`); + return; + } + targets = parsed; + } catch (error) { + core.setFailed(`Failed to parse targets_${profile}_json: ${error.message}`); + return; + } + + if (targets.length === 0) { + targets = [process.env.CURRENT_REPOSITORY]; + core.info(`No configured targets for '${profile}'. Falling back to current repository: ${targets[0]}`); + } + + const targetRef = String(process.env.TARGET_REF || 'main').trim() || 'main'; + const shardCountRaw = String(process.env.SHARD_COUNT || '1').trim(); + const shardIndexRaw = String(process.env.SHARD_INDEX || '0').trim(); + const shardCount = Number.parseInt(shardCountRaw, 10); + const shardIndex = Number.parseInt(shardIndexRaw, 10); + + if (!Number.isInteger(shardCount) || shardCount <= 0) { + core.setFailed(`shard_count must be a positive integer, got '${shardCountRaw}'`); + return; + } + + if (!Number.isInteger(shardIndex) || shardIndex < 0) { + core.setFailed(`shard_index must be a non-negative integer, got '${shardIndexRaw}'`); + return; + } + + core.setOutput('targets_json', JSON.stringify(targets)); + core.setOutput('target_ref', targetRef); + core.setOutput('shard_count', String(shardCount)); + core.setOutput('shard_index', String(shardIndex)); + + select_shard: + needs: [resolve_policy] + runs-on: ubuntu-latest + outputs: + selected_targets_json: ${{ steps.select.outputs.selected_targets_json }} + primary_target_repo: ${{ steps.select.outputs.primary_target_repo }} + steps: + - id: select + uses: actions/github-script@v7 + env: + TARGETS_JSON: ${{ needs.resolve_policy.outputs.targets_json }} + SHARD_COUNT: ${{ needs.resolve_policy.outputs.shard_count }} + SHARD_INDEX: ${{ needs.resolve_policy.outputs.shard_index }} + with: + script: | + const targets = JSON.parse(process.env.TARGETS_JSON || '[]'); + const shardCount = Number.parseInt(process.env.SHARD_COUNT || '1', 10); + const shardIndex = Number.parseInt(process.env.SHARD_INDEX || '0', 10); + + if (!Array.isArray(targets) || targets.length === 0) { + core.setFailed('No target repositories found after policy resolution'); + return; + } + + if (shardIndex >= shardCount) { + core.setFailed(`shard_index (${shardIndex}) must be less than shard_count (${shardCount})`); + return; + } + + const shardSize = Math.ceil(targets.length / shardCount); + const start = shardIndex * shardSize; + const end = start + shardSize; + const selected = targets.slice(start, end); + + if (selected.length === 0) { + core.setFailed(`Selected shard ${shardIndex} is empty for ${targets.length} targets and ${shardCount} shards`); + return; + } + + core.setOutput('selected_targets_json', JSON.stringify(selected)); + core.setOutput('primary_target_repo', selected[0]); + + deterministic_build: + needs: [resolve_policy, select_shard] + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + repo: ${{ fromJson(needs.select_shard.outputs.selected_targets_json) }} + steps: + - name: Checkout selected repository + uses: actions/checkout@v5 + with: + repository: ${{ matrix.repo }} + ref: ${{ needs.resolve_policy.outputs.target_ref }} + token: ${{ secrets.REPO_TOKEN }} + + - name: Deterministic validation + shell: bash + run: | + set -euo pipefail + + echo "Validating repository: ${{ matrix.repo }}" + git rev-parse HEAD + + if [[ -f package.json ]]; then + npm ci + npm run --if-present build + elif [[ -f go.mod ]]; then + go build ./... + else + echo "No default build strategy detected; checkout validation completed." + fi + diff --git a/.github/workflows/reusable-central-replay-failed-shards.yml b/.github/workflows/reusable-central-replay-failed-shards.yml new file mode 100644 index 00000000000..9ac8cd08c99 --- /dev/null +++ b/.github/workflows/reusable-central-replay-failed-shards.yml @@ -0,0 +1,105 @@ +name: Reusable Central Replay Failed Shards + +on: + workflow_call: + inputs: + failed_repos_json: + description: JSON array of owner/repo values that failed in a previous run + required: false + default: "[]" + type: string + target_ref: + description: Git ref used for replay checkout/build validation + required: false + default: main + type: string + max_replays: + description: Maximum number of failed repositories to replay in one run + required: false + default: "10" + type: string + outputs: + replay_targets_json: + description: JSON array of replay targets selected for this run + value: ${{ jobs.select_replays.outputs.replay_targets_json }} + replay_count: + description: Number of replay targets selected + value: ${{ jobs.select_replays.outputs.replay_count }} + +permissions: + contents: read + +jobs: + select_replays: + runs-on: ubuntu-latest + outputs: + replay_targets_json: ${{ steps.select.outputs.replay_targets_json }} + replay_count: ${{ steps.select.outputs.replay_count }} + steps: + - id: select + uses: actions/github-script@v7 + env: + FAILED_REPOS_JSON: ${{ inputs.failed_repos_json }} + MAX_REPLAYS: ${{ inputs.max_replays }} + with: + script: | + const raw = process.env.FAILED_REPOS_JSON || '[]'; + const maxReplaysRaw = String(process.env.MAX_REPLAYS || '10').trim(); + const maxReplays = Number.parseInt(maxReplaysRaw, 10); + + if (!Number.isInteger(maxReplays) || maxReplays <= 0) { + core.setFailed(`max_replays must be a positive integer, got '${maxReplaysRaw}'`); + return; + } + + let repos; + try { + repos = JSON.parse(raw); + } catch (error) { + core.setFailed(`failed_repos_json is not valid JSON: ${error.message}`); + return; + } + + if (!Array.isArray(repos)) { + core.setFailed('failed_repos_json must be a JSON array'); + return; + } + + const cleaned = [...new Set(repos.filter((repo) => typeof repo === 'string' && repo.includes('/')))]; + const selected = cleaned.slice(0, maxReplays); + + core.setOutput('replay_targets_json', JSON.stringify(selected)); + core.setOutput('replay_count', String(selected.length)); + + deterministic_replay: + needs: [select_replays] + if: ${{ needs.select_replays.outputs.replay_count != '0' }} + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + repo: ${{ fromJson(needs.select_replays.outputs.replay_targets_json) }} + steps: + - name: Checkout replay target repository + uses: actions/checkout@v5 + with: + repository: ${{ matrix.repo }} + ref: ${{ inputs.target_ref }} + token: ${{ secrets.REPO_TOKEN }} + + - name: Deterministic replay validation + shell: bash + run: | + set -euo pipefail + + echo "Replaying failed target: ${{ matrix.repo }}" + git rev-parse HEAD + + if [[ -f package.json ]]; then + npm ci + npm run --if-present build + elif [[ -f go.mod ]]; then + go build ./... + else + echo "No default build strategy detected; replay checkout validation completed." + fi diff --git a/.github/workflows/reusable-central-rollout-summary.yml b/.github/workflows/reusable-central-rollout-summary.yml new file mode 100644 index 00000000000..3f9896bdadf --- /dev/null +++ b/.github/workflows/reusable-central-rollout-summary.yml @@ -0,0 +1,111 @@ +name: Reusable Central Rollout Summary + +on: + workflow_call: + inputs: + objective: + description: Rollout objective used for this run + required: false + default: "" + type: string + rollout_profile: + description: Rollout profile (pilot, standard, broad) + required: false + default: "standard" + type: string + selected_targets_json: + description: JSON array of policy-selected targets from deterministic gate + required: false + default: "[]" + type: string + replay_targets_json: + description: JSON array of replay targets for this run + required: false + default: "[]" + type: string + created_prs: + description: Number of PRs created by the rollout workflow + required: false + default: "0" + type: string + created_issues: + description: Number of issues created by the rollout workflow + required: false + default: "0" + type: string + outputs: + summary_json: + description: JSON summary payload for downstream reporting + value: ${{ jobs.summarize.outputs.summary_json }} + +permissions: + contents: read + +jobs: + summarize: + runs-on: ubuntu-latest + outputs: + summary_json: ${{ steps.generate.outputs.summary_json }} + steps: + - id: generate + uses: actions/github-script@v7 + env: + OBJECTIVE: ${{ inputs.objective }} + ROLLOUT_PROFILE: ${{ inputs.rollout_profile }} + SELECTED_TARGETS_JSON: ${{ inputs.selected_targets_json }} + REPLAY_TARGETS_JSON: ${{ inputs.replay_targets_json }} + CREATED_PRS: ${{ inputs.created_prs }} + CREATED_ISSUES: ${{ inputs.created_issues }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + with: + script: | + const parseArray = (raw, name) => { + try { + const parsed = JSON.parse(raw || '[]'); + if (!Array.isArray(parsed)) { + throw new Error(`${name} must be a JSON array`); + } + return parsed.filter((item) => typeof item === 'string'); + } catch (error) { + core.setFailed(`${name} parse error: ${error.message}`); + return null; + } + }; + + const selectedTargets = parseArray(process.env.SELECTED_TARGETS_JSON, 'selected_targets_json'); + const replayTargets = parseArray(process.env.REPLAY_TARGETS_JSON, 'replay_targets_json'); + if (!selectedTargets || !replayTargets) { + return; + } + + const toInt = (raw) => { + const value = Number.parseInt(String(raw || '0').trim(), 10); + return Number.isInteger(value) && value >= 0 ? value : 0; + }; + + const summary = { + objective: String(process.env.OBJECTIVE || '').trim(), + rollout_profile: process.env.ROLLOUT_PROFILE || 'standard', + selected_target_count: selectedTargets.length, + replay_target_count: replayTargets.length, + created_prs: toInt(process.env.CREATED_PRS), + created_issues: toInt(process.env.CREATED_ISSUES), + run_url: process.env.RUN_URL, + generated_at: new Date().toISOString(), + }; + + core.summary + .addHeading('CentralRepoOps Rollout Summary') + .addTable([ + [{ data: 'Field', header: true }, { data: 'Value', header: true }], + ['Objective', summary.objective || '(not provided)'], + ['Profile', summary.rollout_profile], + ['Selected targets', String(summary.selected_target_count)], + ['Replay targets', String(summary.replay_target_count)], + ['PRs created', String(summary.created_prs)], + ['Issues created', String(summary.created_issues)], + ['Run URL', summary.run_url], + ]) + .write(); + + core.setOutput('summary_json', JSON.stringify(summary)); diff --git a/cmd/gh-aw/main.go b/cmd/gh-aw/main.go index 01aed77d7b3..52db681ef19 100644 --- a/cmd/gh-aw/main.go +++ b/cmd/gh-aw/main.go @@ -144,7 +144,7 @@ Examples: return cli.CreateWorkflowInteractively(cmd.Context(), workflowName, verbose, forceFlag) } - // Template mode with workflow name + // Create workflow with workflow name workflowName := args[0] return cli.NewWorkflow(workflowName, verbose, forceFlag) }, diff --git a/docs/src/content/docs/patterns/centralrepoops.md b/docs/src/content/docs/patterns/centralrepoops.md new file mode 100644 index 00000000000..25a11956859 --- /dev/null +++ b/docs/src/content/docs/patterns/centralrepoops.md @@ -0,0 +1,387 @@ +--- +title: CentralRepoOps +description: Operate and roll out changes across many repositories from a single private control repository +sidebar: + badge: { text: 'Advanced', variant: 'caution' } +draft: true +--- + +CentralRepoOps is a [MultiRepoOps](/gh-aw/patterns/multirepoops/) deployment variant where a single private repository acts as a control plane for large-scale operations across many repositories. + +Use this pattern when you need to coordinate rollouts, policy updates, and tracking across tens or hundreds of repositories from one place. + +## Authoring Profiles + +Use profile-based authoring to pick your starting complexity: + +- **Lean profile** for instruction-first rollout workflows with explicit tunable controls in one file. +- **Advanced profile** when you need campaign and wave-promotion controls. + +## Examples + +CentralRepoOps follows the standard single-source model: one `.md` source file plus `.lock.yml` after compile, with deterministic rollout mechanics delegated to a reusable gate workflow. + +### Simple rollout example + +Use this when you want one focused change in one repository per run. + +```aw wrap +--- +on: + workflow_dispatch: + inputs: + target_repo: + required: true + objective: + required: true + +engine: + id: copilot + steps: + - name: Checkout target repository + uses: actions/checkout@v5 + with: + repository: ${{ inputs.target_repo }} + token: ${{ secrets.REPO_TOKEN }} + +permissions: + contents: read + +tools: + github: + mode: remote + toolsets: [repos, issues, pull_requests] + +safe-outputs: + create-issue: + max: 2 + create-pull-request: + max: 1 +--- + +# Simple CentralRepoOps Rollout + +Objective: {{inputs.objective}} + +Target repository: {{inputs.target_repo}} + +Make the smallest safe change needed to satisfy the objective. +If unsafe or blocked, create a tracking issue with clear next steps. +``` + +### Complex rollout example + +Use this when enterprise admins should set policy, controls, and caps while deterministic shard/build mechanics stay hidden. + +```aw wrap +--- +on: + workflow_dispatch: + inputs: + objective: + required: true + rollout_profile: + required: false + max_prs: + required: false + max_issues: + required: false + promote_to_next_wave: + required: false + next_rollout_profile: + required: false + manual-approval: production + +concurrency: + group: "centralrepoops-${{ github.workflow }}-${{ inputs.rollout_profile || 'standard' }}" + cancel-in-progress: false + +rate-limit: + max: 2 + window: 60 + +tracker-id: centralrepoops-rollout + +jobs: + deterministic_gate: + uses: ./.github/workflows/reusable-central-deterministic-gate.yml + secrets: inherit + with: + rollout_profile: ${{ inputs.rollout_profile || 'standard' }} + +engine: + id: copilot + needs: + - deterministic_gate + steps: + - name: Checkout primary target repository + uses: actions/checkout@v5 + with: + repository: ${{ needs.deterministic_gate.outputs.primary_target_repo }} + ref: ${{ needs.deterministic_gate.outputs.target_ref }} + token: ${{ secrets.REPO_TOKEN }} + path: target + +permissions: + contents: read + +tools: + github: + mode: remote + toolsets: [repos, issues, pull_requests, actions] + +safe-outputs: + dispatch-workflow: + workflows: + - my-workflow + max: 1 + create-issue: + max: ${{ inputs.max_issues || '20' }} + create-pull-request: + max: ${{ inputs.max_prs || '5' }} + +--- + +# Complex CentralRepoOps Rollout (Admin-Facing) + +Objective: {{inputs.objective}} + +Profile: {{inputs.rollout_profile}} + +Primary target: ${{ needs.deterministic_gate.outputs.primary_target_repo }} @ ${{ needs.deterministic_gate.outputs.target_ref }} + +Deterministic gate status: ${{ needs.deterministic_gate.result }} + +Execute the objective for deterministic gate-selected targets. + +Prefer one focused PR per repository after deterministic checks pass. + +If blocked, create tracking issues with explicit remediation steps. +``` + +Keep tunable defaults explicit in the main workflow so operators can edit knobs in one place. + +Use implicit defaults when omitted: + +- `rollout_profile=standard` +- `max_prs=5` +- `max_issues=20` +- `promote_to_next_wave=false` +- `next_rollout_profile=""` + +Recommended defaults for initial rollout safety: + +- `rate-limit.max: 2` and `rate-limit.window: 60` reduce rapid repeat invocations while still allowing retries. +- `concurrency.cancel-in-progress: false` preserves run auditability by finishing already-started executions. +- `tracker-id: centralrepoops-rollout` creates a stable marker for filtering and reporting rollout artifacts. + +The reusable workflow (`reusable-central-deterministic-gate.yml`) contains the hidden mechanics: + +> [!IMPORTANT] +> The reusable gate workflow is shipped by default. For advanced rollouts, pass optional policy inputs to the reusable gate call (targets, ref, shard settings). + +- target resolution and validation +- shard selection math +- matrix checkout/build for each selected repository +- outputs such as `primary_target_repo` and `target_ref` + +Admin-facing inputs stay intentionally small: + +- `objective` +- `rollout_profile` +- `max_prs` +- `max_issues` + +Optional steering inputs for wave promotion: + +- `promote_to_next_wave` +- `next_rollout_profile` + +Campaign control inputs (connect campaign policy to orchestration): + +- `campaign_id` +- `campaign_goal` + +`campaign_*` inputs are optional. Use them for multi-wave programs with KPI tracking; for one-off rollouts, leave them empty and run the same architecture without campaign semantics. + +Built-in rollout guardrails stay in frontmatter (not admin inputs): + +- `on.manual-approval` +- `concurrency` +- `rate-limit` +- `tracker-id` + +Everything else (target discovery, sharding, build verification, policy allowlists, refs) remains inside the reusable gate. + +Optional reusable-gate inputs for advanced rollout policy: + +- `targets_pilot_json` +- `targets_standard_json` +- `targets_broad_json` +- `target_ref` +- `shard_count` +- `shard_index` + +Example values: + +- `targets_standard_json: '["my-org/service-a","my-org/service-b","partner-org/service-c"]'` +- `target_ref: main` +- `shard_count: '4'` +- `shard_index: '0'` + +Example reusable workflow interface: + +```yaml wrap +on: + workflow_call: + inputs: + rollout_profile: { type: string, required: false } + outputs: + primary_target_repo: + value: ${{ jobs.select_shard.outputs.primary_target_repo }} + target_ref: + value: ${{ jobs.resolve_policy.outputs.target_ref }} + +jobs: + # resolve_policy, select_shard, deterministic_build (hidden implementation) +``` + +## When to Use CentralRepoOps + +- **Rollout operations** - Apply standard updates (e.g., Dependabot config, CI policy, labels) across many repos +- **Central governance** - Enforce organization-wide automation with explicit controls +- **Phased adoption** - Roll out by shard or tier (pilot → broader rollout) +- **Operational reliability** - Retry and replay failures without rerunning successful repos + +## Architecture + +```mermaid +flowchart TD + A[Trigger] --> B[Deterministic Gate] + B --> C[Agentic Control Plane] + C --> D[Deterministic Execution] + + D --> R1[Repo 1] + D --> R2[Repo 2] + D --> R3[...] + D --> RN[Repo N] + + R1 --> E{Stop Criteria} + R2 --> E + R3 --> E + RN --> E + + E -- No --> C + E -- Yes --> F[Run Complete] +``` + +## Canonical Pattern (Slim + Flexible) + +Treat this as the default architecture for CentralRepoOps: + +1. **Slim control workflow** (policy + intent only) + - Keep only admin-facing inputs (`objective`, `rollout_profile`, caps) + - Call the reusable deterministic gate + - Run the agentic step against gate outputs +2. **Reusable deterministic gate** (all mechanics) + - Target resolution + - Shard selection + - Deterministic checkout/build validation + - Stable outputs contract +3. **Optional add-on workflows** (only when needed) + - Replay failed shards + - Promotion waves (`pilot` → `standard` → `broad`) + - Reporting/aggregation + +This keeps the default path simple while still letting you cover advanced campaign scenarios by composition. + +## Optional Add-ons (Shipped) + +These reusable workflows are available for scale operations when you need them: + +- `.github/workflows/reusable-central-replay-failed-shards.yml` +- `.github/workflows/reusable-central-rollout-summary.yml` + +### Replay failed shards + +Use replay after a broad run to retry only failed targets: + +```yaml wrap +jobs: + replay_failed: + uses: ./.github/workflows/reusable-central-replay-failed-shards.yml + secrets: inherit + with: + failed_repos_json: ${{ needs.collect_failures.outputs.failed_repos_json }} + target_ref: ${{ needs.deterministic_gate.outputs.target_ref }} + max_replays: "10" +``` + +### Promotion waves + +For phased rollout (`pilot` → `standard` → `broad`), run the same control workflow three times with different `rollout_profile` values. +Use `safe-outputs.dispatch-workflow` to trigger the next wave only after success and review. + +Example dispatch payload for promoting the next wave: + +```json wrap +{ + "type": "dispatch_workflow", + "workflow_name": "central-repo-ops", + "inputs": { + "objective": "{{inputs.objective}}", + "rollout_profile": "{{inputs.next_rollout_profile}}", + "max_prs": "{{inputs.max_prs}}", + "max_issues": "{{inputs.max_issues}}", + "campaign_id": "{{inputs.campaign_id}}", + "campaign_goal": "{{inputs.campaign_goal}}", + "promote_to_next_wave": "false", + "next_rollout_profile": "" + } +} +``` + +### Rollout summary reporting + +Use a summary job at the end of each run: + +```yaml wrap +jobs: + rollout_summary: + uses: ./.github/workflows/reusable-central-rollout-summary.yml + with: + objective: ${{ inputs.objective }} + rollout_profile: ${{ inputs.rollout_profile }} + replay_targets_json: ${{ needs.replay_failed.outputs.replay_targets_json }} + created_prs: "${{ inputs.max_prs }}" + created_issues: "${{ inputs.max_issues }}" +``` + +`selected_targets_json` is optional in the summary workflow. Include it only if your custom deterministic layer publishes that list. + +## Recommended Execution Model + +Use a hybrid model: + +1. **Agentic control-plane decisions** for steering (next wave, retry scope, mutation intent) +2. **Deterministic workflow backbone** for execution guarantees (fan-out, shard/build validation, approvals, auditable state transitions) + +This keeps large-scale rollout operations reliable while still enabling AI-assisted orchestration. + +## Security & Control Defaults + +- Prefer GitHub App auth over long-lived PATs +- Use explicit `allowed-repos` for cross-repo writes +- Keep conservative `max` limits in safe outputs +- Use phased rollout (`5 → 25 → 100+ repos`) before org-wide execution + +## Example Use Cases + +- Roll out `.github/dependabot.yml` to all repos in a shard +- Create tracking issues for stale CI failures across multiple repos +- Apply standardized labels and branch policy helper PRs across teams + +## Related Patterns + +- [MultiRepoOps](/gh-aw/patterns/multirepoops/) - Cross-repository operations concept +- [SideRepoOps](/gh-aw/patterns/siderepoops/) - Separate automation repo pattern diff --git a/pkg/cli/commands.go b/pkg/cli/commands.go index 5b29bc4b669..f183cadf917 100644 --- a/pkg/cli/commands.go +++ b/pkg/cli/commands.go @@ -235,29 +235,28 @@ func NewWorkflow(workflowName string, verbose bool, force bool) error { return fmt.Errorf("failed to create .github/workflows directory: %w", err) } - // Construct the destination file path - destFile := filepath.Join(githubWorkflowsDir, workflowName+".md") - commandsLog.Printf("Destination file: %s", destFile) - - // Validate the destination file path - destFile, err = fileutil.ValidateAbsolutePath(destFile) - if err != nil { - commandsLog.Printf("Invalid destination file path: %v", err) - return fmt.Errorf("invalid destination file path: %w", err) + filesToWrite := createWorkflowTemplateFiles(workflowName, githubWorkflowsDir) + for i := range filesToWrite { + validatedPath, validateErr := fileutil.ValidateAbsolutePath(filesToWrite[i].Path) + if validateErr != nil { + commandsLog.Printf("Invalid destination file path: %v", validateErr) + return fmt.Errorf("invalid destination file path: %w", validateErr) + } + filesToWrite[i].Path = validatedPath } - // Check if destination file already exists - if _, err := os.Stat(destFile); err == nil && !force { - commandsLog.Printf("Workflow file already exists and force=false: %s", destFile) - return fmt.Errorf("workflow file '%s' already exists. Use --force to overwrite", destFile) - } + destFile := filesToWrite[0].Path + commandsLog.Printf("Destination file: %s", destFile) - // Create the template content - template := createWorkflowTemplate(workflowName) + if err := ensureTemplateTargetsAvailable(filesToWrite, force); err != nil { + commandsLog.Printf("Template target check failed: %v", err) + return err + } - // Write the template to file with restrictive permissions (owner-only) - if err := os.WriteFile(destFile, []byte(template), 0600); err != nil { - return fmt.Errorf("failed to write workflow file '%s': %w", destFile, err) + for _, file := range filesToWrite { + if err := writeTemplateFile(file); err != nil { + return err + } } fmt.Fprintln(os.Stderr, console.FormatSuccessMessage(fmt.Sprintf("Created new workflow: %s", destFile))) @@ -266,57 +265,34 @@ func NewWorkflow(workflowName string, verbose bool, force bool) error { return nil } -// createWorkflowTemplate generates a concise workflow template with essential options -func createWorkflowTemplate(workflowName string) string { - return `--- -# Trigger - when should this workflow run? -on: - workflow_dispatch: # Manual trigger - -# Alternative triggers (uncomment to use): -# on: -# issues: -# types: [opened, reopened] -# pull_request: -# types: [opened, synchronize] -# schedule: daily # Fuzzy daily schedule (scattered execution time) -# # schedule: weekly on monday # Fuzzy weekly schedule - -# Permissions - what can this workflow access? -permissions: - contents: read - issues: write - pull-requests: write - -# Outputs - what APIs and tools can the AI use? -safe-outputs: - create-issue: # Creates issues (default max: 1) - max: 5 # Optional: specify maximum number - # create-agent-session: # Creates GitHub Copilot coding agent sessions (max: 1) - # create-pull-request: # Creates exactly one pull request - # add-comment: # Adds comments (default max: 1) - # max: 2 # Optional: specify maximum number - # add-labels: - ---- - -# ` + workflowName + ` - -Describe what you want the AI to do when this workflow runs. - -## Instructions +type workflowTemplateFile struct { + Path string + Content string + Mode os.FileMode +} -Replace this section with specific instructions for the AI. For example: +func ensureTemplateTargetsAvailable(files []workflowTemplateFile, force bool) error { + for _, file := range files { + if _, err := os.Stat(file.Path); err == nil && !force { + if strings.HasSuffix(file.Path, ".md") { + return fmt.Errorf("workflow file '%s' already exists. Use --force to overwrite", file.Path) + } + return fmt.Errorf("template target '%s' already exists. Use --force to overwrite", file.Path) + } + } -1. Read the issue description and comments -2. Analyze the request and gather relevant information -3. Provide a helpful response or take appropriate action + return nil +} -Be clear and specific about what the AI should accomplish. +func writeTemplateFile(file workflowTemplateFile) error { + dir := filepath.Dir(file.Path) + if err := os.MkdirAll(dir, 0755); err != nil { + return fmt.Errorf("failed to create directory '%s': %w", dir, err) + } -## Notes + if err := os.WriteFile(file.Path, []byte(file.Content), file.Mode); err != nil { + return fmt.Errorf("failed to write template file '%s': %w", file.Path, err) + } -- Run ` + "`" + string(constants.CLIExtensionPrefix) + " compile`" + ` to generate the GitHub Actions workflow -- See https://github.github.com/gh-aw/ for complete configuration options and tools documentation -` + return nil } diff --git a/pkg/cli/commands_test.go b/pkg/cli/commands_test.go index ab749429ea4..1daf38dfa26 100644 --- a/pkg/cli/commands_test.go +++ b/pkg/cli/commands_test.go @@ -603,13 +603,14 @@ func TestNewWorkflow(t *testing.T) { contentStr := string(content) // Check for key template elements expectedElements := []string{ - "# Trigger - when should this workflow run?", "on:", "permissions:", "safe-outputs:", "# " + normalizedName, "workflow_dispatch:", } + + expectedElements = append(expectedElements, "# Trigger - when should this workflow run?") for _, element := range expectedElements { if !strings.Contains(contentStr, element) { t.Errorf("Template missing expected element: %s", element) @@ -620,6 +621,8 @@ func TestNewWorkflow(t *testing.T) { // Clean up for next test os.RemoveAll(".github") + os.RemoveAll("inventory") + os.RemoveAll("scripts") }) } } diff --git a/pkg/cli/workflow_templates.go b/pkg/cli/workflow_templates.go new file mode 100644 index 00000000000..873f123ca67 --- /dev/null +++ b/pkg/cli/workflow_templates.go @@ -0,0 +1,38 @@ +package cli + +import ( + _ "embed" + "path/filepath" + "strings" + + "github.com/github/gh-aw/pkg/constants" +) + +const ( + workflowNamePlaceholder = "__WORKFLOW_NAME__" + cliPrefixPlaceholder = "__CLI_PREFIX__" +) + +//go:embed workflowtemplates/default.md.tmpl +var defaultWorkflowTemplate string + +func renderWorkflowTemplate(raw string, workflowName string) string { + replacer := strings.NewReplacer( + workflowNamePlaceholder, workflowName, + cliPrefixPlaceholder, string(constants.CLIExtensionPrefix), + ) + return replacer.Replace(raw) +} + +// createWorkflowTemplate generates a concise workflow template with essential options +func createWorkflowTemplate(workflowName string) string { + return renderWorkflowTemplate(defaultWorkflowTemplate, workflowName) +} + +func createWorkflowTemplateFiles(workflowName string, githubWorkflowsDir string) []workflowTemplateFile { + files := []workflowTemplateFile{ + {Path: filepath.Join(githubWorkflowsDir, workflowName+".md"), Content: createWorkflowTemplate(workflowName), Mode: 0600}, + } + + return files +} diff --git a/pkg/cli/workflow_templates_test.go b/pkg/cli/workflow_templates_test.go new file mode 100644 index 00000000000..a7e94cc40c5 --- /dev/null +++ b/pkg/cli/workflow_templates_test.go @@ -0,0 +1,48 @@ +//go:build !integration + +package cli + +import ( + "strings" + "testing" +) + +func TestCreateWorkflowTemplateDefault(t *testing.T) { + content := createWorkflowTemplate("my-workflow") + + expectedMarkers := []string{ + "# my-workflow", + "# Trigger - when should this workflow run?", + "permissions:", + "safe-outputs:", + "## Instructions", + "workflow_dispatch:", + } + + for _, marker := range expectedMarkers { + if !strings.Contains(content, marker) { + t.Errorf("default template missing marker: %s", marker) + } + } + + nonExpectedMarkers := []string{ + "promote_to_next_wave:", + "next_rollout_profile:", + "reusable-central-deterministic-gate.yml", + "centralrepoops-", + "imports:", + } + + for _, marker := range nonExpectedMarkers { + if strings.Contains(content, marker) { + t.Errorf("default template should not contain marker: %s", marker) + } + } +} + +func TestCreateWorkflowTemplateFilesMainOnly(t *testing.T) { + files := createWorkflowTemplateFiles("my-workflow", ".github/workflows") + if len(files) != 1 { + t.Fatalf("expected 1 file for workflow template, got %d", len(files)) + } +} diff --git a/pkg/cli/workflowtemplates/default.md.tmpl b/pkg/cli/workflowtemplates/default.md.tmpl new file mode 100644 index 00000000000..184ff631a9c --- /dev/null +++ b/pkg/cli/workflowtemplates/default.md.tmpl @@ -0,0 +1,50 @@ +--- +# Trigger - when should this workflow run? +on: + workflow_dispatch: # Manual trigger + +# Alternative triggers (uncomment to use): +# on: +# issues: +# types: [opened, reopened] +# pull_request: +# types: [opened, synchronize] +# schedule: daily # Fuzzy daily schedule (scattered execution time) +# # schedule: weekly on monday # Fuzzy weekly schedule + +# Permissions - what can this workflow access? +permissions: + contents: read + issues: write + pull-requests: write + +# Outputs - what APIs and tools can the AI use? +safe-outputs: + create-issue: # Creates issues (default max: 1) + max: 5 # Optional: specify maximum number + # create-agent-session: # Creates GitHub Copilot coding agent sessions (max: 1) + # create-pull-request: # Creates exactly one pull request + # add-comment: # Adds comments (default max: 1) + # max: 2 # Optional: specify maximum number + # add-labels: + +--- + +# __WORKFLOW_NAME__ + +Describe what you want the AI to do when this workflow runs. + +## Instructions + +Replace this section with specific instructions for the AI. For example: + +1. Read the issue description and comments +2. Analyze the request and gather relevant information +3. Provide a helpful response or take appropriate action + +Be clear and specific about what the AI should accomplish. + +## Notes + +- Run `__CLI_PREFIX__ compile` to generate the GitHub Actions workflow +- See https://github.github.com/gh-aw/ for complete configuration options and tools documentation