diff --git a/.github/workflows/shared/apm.md b/.github/workflows/shared/apm.md index 1d2b7b396..f528df123 100644 --- a/.github/workflows/shared/apm.md +++ b/.github/workflows/shared/apm.md @@ -2,55 +2,244 @@ # APM (Agent Package Manager) - Shared Workflow # Install Microsoft APM packages in your agentic workflow. # -# This shared workflow creates a dedicated "apm" job (depending on activation) that -# packs packages using microsoft/apm-action and uploads the bundle as an artifact. -# The agent job then downloads and unpacks the bundle as pre-steps. +# This shared workflow normalises packages, single-app inputs, and apps[] (multi-org +# GitHub App credential groups) into one canonical list of credential groups in an +# "apm-prep" job, then fans the "apm" job out one matrix replica per group. Each +# replica mints its own installation token (when an app-id is set), packs only its +# declared packages with microsoft/apm-action, and uploads a uniquely-named artifact. +# Pre-agent-steps then download all bundles and restore them in one apm-action call. +# +# STATUS: blocked on upstream microsoft/apm-action gaining a `bundles-file:` input. +# The matrix restore block in `steps:` below is intentionally commented out until +# that input ships -- see TODO marker. Until then this workflow does not produce +# a working agent run; the diff is for design review only. # # Documentation: https://github.com/microsoft/APM # -# Usage: -# imports: -# - uses: shared/apm.md -# with: -# packages: -# - microsoft/apm-sample-package -# - github/awesome-copilot/skills/review-and-refactor +# Three user-facing forms (all valid, additive): +# +# 1. Public + default-token packages (no App credentials): +# +# imports: +# - uses: shared/apm.md +# with: +# packages: +# - microsoft/apm-sample-package +# - github/awesome-copilot/skills/review-and-refactor +# +# 2. Single GitHub App (one org) -- canonical shorthand: +# +# imports: +# - uses: shared/apm.md +# with: +# app-id: ${{ vars.APP_ID }} +# private-key: ${{ secrets.APP_PRIVATE_KEY }} +# owner: my-org +# packages: +# - my-org/my-private-skills +# +# 3. Multiple GitHub Apps (cross-org): +# +# imports: +# - uses: shared/apm.md +# with: +# packages: +# - microsoft/apm-sample-package +# apps: +# - id: acme +# app-id: ${{ vars.ACME_APP_ID }} +# private-key: ${{ secrets.ACME_KEY }} +# owner: acme-org +# packages: +# - acme-org/acme-skills/skills/code-review +# - app-id: ${{ vars.BETA_APP_ID }} +# private-key: ${{ secrets.BETA_KEY }} +# owner: beta-org +# packages: +# - beta-org/beta-pkg import-schema: packages: type: array items: type: string - required: true + required: false description: > - List of APM package references to install. + Public APM packages or packages reachable via the default token cascade + (GH_AW_PLUGINS_TOKEN, GH_AW_GITHUB_TOKEN, GITHUB_TOKEN). Optional. At + least one of `packages`, the single-app inputs, or `apps` must be provided. Format: owner/repo or owner/repo/path/to/skill. - Examples: microsoft/apm-sample-package, github/awesome-copilot/skills/review-and-refactor + + # Single-app convenience form (canonical shorthand for one-org users) + app-id: + type: string + required: false + description: > + GitHub App ID. With `private-key`, mints an installation token for the + packages listed in `packages:`. For multiple orgs, use `apps:` instead. + private-key: + type: string + required: false + description: > + PEM private key matching `app-id`. Required when `app-id` is set. Pass via + a repository or organization secret. + owner: + type: string + required: false + description: > + App installation owner. Defaults to the current repository owner when + omitted. Only used when `app-id` is set. + repositories: + type: string + required: false + description: > + Repositories the minted token is scoped to. Comma- or newline-separated. + Empty defaults to the calling repo or the App installation default scope. + Note: literal "*" is NOT a wildcard for actions/create-github-app-token; + leave empty for org-wide access via App installation config. + + # Multi-app form (cross-org) + apps: + type: array + required: false + description: > + List of GitHub App credential groups. Each entry mints its own + installation token and packs its own packages. Use when packages span + multiple orgs requiring different App installations. + items: + type: object + properties: + id: + type: string + required: false + description: > + Stable identifier used for matrix-row and artifact naming. + Auto-derived from `owner` (slugified) when omitted. Required when + two entries share the same owner. + app-id: + type: string + required: true + private-key: + type: string + required: true + owner: + type: string + required: false + repositories: + type: string + required: false + packages: + type: array + items: + type: string + required: true jobs: - apm: + apm-prep: runs-on: ubuntu-slim needs: [activation] permissions: {} + outputs: + matrix: ${{ steps.compute.outputs.matrix }} steps: - - name: Prepare APM package list - id: apm_prep + # SECURITY (S3): never echo $groups, $matrix, or any matrix.group.* value + # in any apm-prep step. private-key is a real secret string here. + - name: Compute APM credential-group matrix + id: compute env: - AW_APM_PACKAGES: '${{ github.aw.import-inputs.packages }}' + AW_APM_PACKAGES: ${{ github.aw.import-inputs.packages }} + AW_APM_APPS: ${{ github.aw.import-inputs.apps }} + AW_APM_LEGACY_APP_ID: ${{ github.aw.import-inputs.app-id }} + AW_APM_LEGACY_PRIVATE_KEY: ${{ github.aw.import-inputs.private-key }} + AW_APM_LEGACY_OWNER: ${{ github.aw.import-inputs.owner }} + AW_APM_LEGACY_REPOS: ${{ github.aw.import-inputs.repositories }} run: | - DEPS=$(echo "$AW_APM_PACKAGES" | jq -r '.[] | "- " + .') + set -euo pipefail + packages_json=${AW_APM_PACKAGES:-null} + apps_json=${AW_APM_APPS:-null} + legacy_id=${AW_APM_LEGACY_APP_ID:-} + + groups=$(jq -nc \ + --argjson packages "$packages_json" \ + --argjson apps "$apps_json" \ + --arg legacy_id "$legacy_id" \ + --arg legacy_pk "${AW_APM_LEGACY_PRIVATE_KEY:-}" \ + --arg legacy_owner "${AW_APM_LEGACY_OWNER:-}" \ + --arg legacy_repos "${AW_APM_LEGACY_REPOS:-}" \ + 'def slug(s): s | gsub("[^a-zA-Z0-9-]"; "-") | ascii_downcase | .[0:32]; + def with_id(g): + g + (if (g.id // "") == "" then {id: ("auto-" + slug(g.owner // "default"))} else {} end); + [ + (if (($packages // []) | length) > 0 and $legacy_id == "" + then [{id:"default",("app-id"):"",("private-key"):"",owner:"",repositories:"",packages:$packages}] + else [] end), + (if $legacy_id != "" + then [with_id({id:"legacy",("app-id"):$legacy_id,("private-key"):$legacy_pk,owner:$legacy_owner,repositories:$legacy_repos,packages:($packages // [])})] + else [] end), + (($apps // []) | map(with_id(.))) + ] | add // []') + + count=$(echo "$groups" | jq 'length') + if [ "$count" = "0" ]; then + echo "::error::shared/apm.md import provided no packages. Add packages: , single-app inputs (app-id + private-key), or apps: in the with: block." + exit 1 + fi + + dups=$(echo "$groups" | jq -r '[.[].id] | group_by(.) | map(select(length > 1) | first) | join(", ")') + if [ -n "$dups" ]; then + echo "::error::duplicate apm group ids after auto-derivation: $dups. Set apps[].id explicitly when two entries share the same owner." + exit 1 + fi + + while IFS= read -r id; do + if ! echo "$id" | grep -Eq '^[a-z0-9-]{1,32}$'; then + echo "::error::invalid apm group id: '$id' (lowercase alphanumeric and dashes, 1-32 chars). Set apps[].id explicitly." + exit 1 + fi + done < <(echo "$groups" | jq -r '.[].id') + + # SAFE: emit only id + package-count to logs. Never $groups in full. + { + echo "matrix={\"group\":$groups}" + } >> "$GITHUB_OUTPUT" + printf "::notice::APM matrix: %d credential group(s)\n" "$count" + echo "$groups" | jq -r '.[] | " - " + .id + " (" + (.packages | length | tostring) + " package(s))"' + + apm: + runs-on: ubuntu-slim + needs: [activation, apm-prep] + permissions: {} + strategy: + fail-fast: false + matrix: ${{ fromJSON(needs.apm-prep.outputs.matrix) }} + steps: + - name: Mint installation token + id: token + if: ${{ matrix.group.app-id != '' }} + uses: actions/create-github-app-token@v3.1.1 + with: + app-id: ${{ matrix.group.app-id }} + private-key: ${{ matrix.group.private-key }} + owner: ${{ matrix.group.owner != '' && matrix.group.owner || github.repository_owner }} + repositories: ${{ matrix.group.repositories }} + - name: Render package list + id: list + env: + AW_PKG: ${{ toJSON(matrix.group.packages) }} + run: | + DEPS=$(echo "$AW_PKG" | jq -r '.[] | "- " + .') { echo "deps<> "$GITHUB_OUTPUT" - name: Pack APM packages - id: apm_pack - uses: microsoft/apm-action@v1.4.2 + id: pack + uses: microsoft/apm-action@v1.5.0 env: - GITHUB_TOKEN: ${{ secrets.GH_AW_PLUGINS_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + GITHUB_TOKEN: ${{ steps.token.outputs.token || secrets.GH_AW_PLUGINS_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} with: - dependencies: ${{ steps.apm_prep.outputs.deps }} + dependencies: ${{ steps.list.outputs.deps }} isolated: 'true' pack: 'true' archive: 'true' @@ -60,46 +249,81 @@ jobs: if: success() uses: actions/upload-artifact@v7 with: - name: ${{ needs.activation.outputs.artifact_prefix }}apm - path: ${{ steps.apm_pack.outputs.bundle-path }} + name: ${{ needs.activation.outputs.artifact_prefix }}apm-${{ matrix.group.id }} + path: ${{ steps.pack.outputs.bundle-path }} retention-days: '1' steps: - - name: Download APM bundle artifact + - name: Download APM bundle artifacts (all groups) uses: actions/download-artifact@v8.0.1 with: - name: ${{ needs.activation.outputs.artifact_prefix }}apm - path: /tmp/gh-aw/apm-bundle - - name: Find APM bundle path - id: apm_bundle - run: echo "path=$(find /tmp/gh-aw/apm-bundle -name '*.tar.gz' | head -1)" >> "$GITHUB_OUTPUT" - - name: Restore APM packages - uses: microsoft/apm-action@v1.4.2 + pattern: ${{ needs.activation.outputs.artifact_prefix }}apm-* + path: /tmp/gh-aw/apm-bundles + merge-multiple: false + - name: Validate downloaded bundles match matrix manifest + env: + EXPECTED_MATRIX: ${{ needs.apm-prep.outputs.matrix }} + ARTIFACT_PREFIX: ${{ needs.activation.outputs.artifact_prefix }} + run: | + set -euo pipefail + expected=$(echo "$EXPECTED_MATRIX" | jq -r --arg prefix "$ARTIFACT_PREFIX" '.group | map($prefix + "apm-" + .id) | sort | .[]') + actual=$(ls /tmp/gh-aw/apm-bundles | sort) + missing=$(comm -23 <(echo "$expected") <(echo "$actual") || true) + unexpected=$(comm -13 <(echo "$expected") <(echo "$actual") || true) + if [ -n "$missing" ]; then + echo "::error::missing APM bundles (group did not pack successfully): $missing" + exit 1 + fi + if [ -n "$unexpected" ]; then + echo "::error::unexpected artifact in apm bundle download (collision attack?): $unexpected" + exit 1 + fi + - name: Build bundle list + id: bundles + run: | + set -euo pipefail + mapfile -t list < <(find /tmp/gh-aw/apm-bundles -name '*.tar.gz' | sort) + [ ${#list[@]} -gt 0 ] || { echo '::error::no apm bundles found'; exit 1; } + printf '%s\n' "${list[@]}" > /tmp/gh-aw/apm-bundle-list.txt + - name: Restore APM packages (all bundles) + uses: microsoft/apm-action@v1.5.0 with: - bundle: ${{ steps.apm_bundle.outputs.path }} + bundles-file: /tmp/gh-aw/apm-bundle-list.txt --- diff --git a/CHANGELOG.md b/CHANGELOG.md index 899629591..44f95f87b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- `shared/apm.md` gh-aw workflow gains an `apps:` array input for cross-org private packages: each entry mints its own GitHub App installation token via `actions/create-github-app-token` and packs only its declared packages, with a matrix fan-out one replica per credential group. The single-app top-level form (`app-id`, `private-key`, `owner`, `repositories`) shipped earlier in this cycle is preserved as the canonical shorthand for one-org users; `apps[]` is purely additive. Multi-bundle restore uses the `bundles-file:` input from `microsoft/apm-action@v1.5.0` (microsoft/apm-action#30, microsoft/apm-action#29). +- `shared/apm.md` gh-aw workflow now accepts `app-id`, `private-key`, `owner`, and `repositories` inputs to mint a GitHub App installation token for fetching cross-org private APM packages, restoring parity with the deprecated `dependencies.github-app` form. The default `GH_AW_PLUGINS_TOKEN || GH_AW_GITHUB_TOKEN || GITHUB_TOKEN` cascade still applies when no app-id is supplied. + ### Changed - **Manifest contract: invalid `target:` values now raise a parse error.** Previously, an unknown token (or a CSV string like `target: opencode,claude,copilot,agents` instead of the YAML list `target: [opencode, claude, copilot, agents]`) was silently ignored, leaving `apm install` and `apm compile` to exit 0 while deploying nothing. The shared parser used by `--target` now also validates `apm.yml`'s `target:`, so the same input resolves the same way at every entry point. **Migration:** three previously-silent inputs now fail loud -- (1) unknown tokens (`target: bogus` -> fix the typo), (2) empty values (`target: ""`, `target: []` -> remove the line if you meant auto-detect), (3) `all` mixed with other targets (`target: [all, claude]` -> use `all` alone). Omitting `target:` entirely still triggers auto-detection. (#820)