Skip to content

pre-steps outputs unavailable to safe_outputs/conclusion/activation jobs that mint GitHub App tokens #26719

@bbonafed

Description

@bbonafed

Summary

pre-steps: (v0.67.3) was shipped in response to #18542 and #25710 as the designated path for fetching credentials from external secret managers (Conjur, Vault, AWS Secrets Manager, etc.) before the agent runs. However, pre-steps: is injected only into the agent job. The compiled .lock.yml also mints GitHub App tokens in safe_outputs, conclusion, and activation jobs — and those jobs receive no equivalent pre-step, so custom step outputs like ${{ steps.get-secrets.outputs.* }} cannot feed their actions/create-github-app-token invocations.

The net effect: pre-steps: + an external secret manager can supply app credentials to the agent / MCP token, but the app's app-id and private-key still have to live in ${{ secrets.* }} for every other job. This is the gap the original #18542 problem statement called out ("The compiled .lock.yml references ${{ secrets.* }} across multiple independent jobs (activation, agent, safe-outputs, conclusion). Since each job runs on a separate runner, a secret-fetching step in one job cannot make values available to the others..."), but that scope was only partially delivered.

This issue requests closing that remaining gap.

Agent Analysis

I reproduced the behavior with a spike workflow that declares pre-steps: using a hypothetical org-internal/secrets@v2 action (a wrapper around an external secret manager) and references its outputs in both tools.github.github-app and safe-outputs.github-app:

pre-steps:
  - name: Get App Credentials
    id: get-secrets
    uses: org-internal/secrets@v2
    with:
      username: ${{ secrets.SECRET_MANAGER_USERNAME }}
      password: ${{ secrets.SECRET_MANAGER_PASSWORD }}
      secrets_to_retrieve: |
        WORKFLOW_APP_ID,
        WORKFLOW_APP_PRIVATE_KEY

tools:
  github:
    github-app:
      app-id: ${{ steps.get-secrets.outputs.WORKFLOW_APP_ID }}
      private-key: ${{ steps.get-secrets.outputs.WORKFLOW_APP_PRIVATE_KEY }}

safe-outputs:
  github-app:
    app-id: ${{ steps.get-secrets.outputs.WORKFLOW_APP_ID }}
    private-key: ${{ steps.get-secrets.outputs.WORKFLOW_APP_PRIVATE_KEY }}
  # ... safe output handlers ...

Compilation result

Compiles cleanly with gh-aw v0.68.4. The expression validator (pkg/workflow/expression_safety_validation.go, needsStepsRegex = ^(needs|steps)\.[a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)*$) accepts steps.get-secrets.outputs.* in both tools.github.github-app.private-key and safe-outputs.github-app.private-key. Good.

Generated .lock.yml (the problem)

Job Mints actions/create-github-app-token? Has get-secrets pre-step injected? Result
agent yes (github-mcp-app-token) ✅ yes Works — same-job step reference resolves
safe_outputs yes (safe-outputs-app-token) ❌ no Broken at runtimeapp-id / private-key evaluate to empty string
conclusion yes (safe-outputs-app-token for safe-output API calls) ❌ no Broken at runtime
activation yes in workflows with on: issues / on: schedule triggers needing reactions / skip-if-no-match ❌ no Broken at runtime
pre_activation no n/a n/a
detection no n/a n/a

The compiler correctly substitutes the steps.get-secrets.outputs.* references into every create-github-app-token step it emits (in all four jobs above), but because the get-secrets step is only present in the agent job, the three non-agent jobs reference a step that doesn't exist in their scope and those token-minting steps fail at runtime.

Root Cause

pre-steps: is emitted into a single job during compilation, not duplicated into sibling jobs that also need its outputs. Relevant source locations:

  1. pkg/workflow/workflow_github_app.goapplyTopLevelGitHubAppFallbacks walks all token-minting sites (activation, safe-outputs, checkout, tools.github) and applies the resolved GitHubAppConfig to each, but the PrivateKey / AppID strings stored in GitHubAppConfig are literal expression strings (e.g. ${{ steps.get-secrets.outputs.WORKFLOW_APP_PRIVATE_KEY }}). Those strings are rendered into create-github-app-token steps in every dependent job with no awareness that the referenced steps.* is only defined in one job.

  2. pkg/workflow/compiler_main_job.go (and siblings) — pre-steps: content is spliced into the agent job's step list. No corresponding splice happens for safe_outputs, conclusion, or activation jobs.

  3. v0.67.3 blog post explicitly scopes the feature to a single job: "inject steps that run before checkout and the agent inside the same job". The docs at docs/src/content/docs/reference/frontmatter-full.md further scope the recommended consumer to checkout.token only. Neither documents the cross-job gap.

Why the Workarounds Fall Short

Approach Problem
Store app private-key in ${{ secrets.* }} and keep pre-steps: for the agent job only The private key is exactly what external-secret-manager users are trying to avoid storing in GitHub Secrets. This re-introduces the policy violation #18542 set out to solve.
Move secret fetching to a separate jobs: entry and pass via needs.*.outputs.* GitHub Actions masks secret-manager action outputs across job boundaries (see #24897, #25122 for the same class of failure with checkout.token). The downstream job receives *** instead of real values.
Wrap the gh-aw workflow in a workflow_call reusable with a caller that fetches secrets and passes them as secrets: inputs Works, but forces every agent workflow to grow a caller wrapper plus a workflow_call trigger purely for credential plumbing. Adds indirection, breaks direct workflow_dispatch UX, and doubles the CI surface area.
strict: false + plain GitHub Secrets Disables unrelated strict-mode protections.

Proposed Solutions

Option A (preferred): duplicate pre-steps: into every job that mints a GitHub App token

Make pre-steps: a job-scoped concept that is automatically replicated into any job the compiler emits which contains a create-github-app-token step (or, more generally, which references steps.<id>.outputs.* from a pre-steps step id). Current token-minting sites that need it:

  • agent (already has it)
  • safe_outputs
  • conclusion (for safe-output API calls)
  • activation (when reactions / skip-if-no-match / status-comment need an app token)

Trade-off: the external-secret-manager action runs up to 4× per workflow run. For most enterprise secret managers this is acceptable (sub-second calls, well within rate limits), but it should be opt-out-able.

Option B: a new top-level pre-steps-all-jobs: (or a flag on pre-steps: entries)

Explicit opt-in so users who only need per-agent secrets (the current behavior) don't pay the replication cost:

pre-steps:
  - name: Get App Credentials
    id: get-secrets
    scope: all-token-mint-jobs   # new
    uses: org-internal/secrets@v2
    with: ...

Or a sibling field:

pre-steps-all-jobs:
  - id: get-secrets
    uses: org-internal/secrets@v2
    with: ...

Option C: validate and fail fast at compile time

If A/B are not in scope, at minimum the compiler should detect that a steps.<id>.outputs.* reference in safe-outputs.github-app / activation on.github-app / conclusion does not resolve within the target job and produce a clear compile-time error instead of a runtime failure. This turns today's silent footgun into a loud, actionable compile error.

Implementation Plan (agent-ready, preferring Option A)

  1. Identify token-mint job set (pkg/workflow/workflow_github_app.go):

    • Extend applyTopLevelGitHubAppFallbacks (or introduce a sibling helper) to return the set of job IDs that will emit a create-github-app-token step: {agent, safe_outputs, conclusion, activation} (conditional on which sections are configured).
  2. Introduce a pre-steps replicator (new file, e.g. pkg/workflow/pre_steps_replication.go):

    • Accepts WorkflowData.PreSteps and the token-mint job set.
    • Emits the same step list into each targeted job's step array, preserving order relative to the existing Setup Scripts / Generate GitHub App token steps (pre-steps should run before the token-mint step in each job).
  3. Wire into job compilers:

    • compiler_main_job.go (agent job) — no-op; already correct.
    • compiler_safe_outputs_job.go — splice pre-steps before the safe-outputs-app-token generation step.
    • compiler_conclusion_job.go — same.
    • compiler_activation_job.go — splice before activation-app-token generation (when activation minting is enabled).
  4. Schema / validation:

  5. Docs (docs/src/content/docs/reference/frontmatter-full.md, docs/src/content/docs/reference/triggers.md):

    • Document the replication behavior and the scope: field.
    • Add an end-to-end example using a generic external secret manager feeding tools.github.github-app and safe-outputs.github-app.
  6. Tests:

    • pkg/workflow/pre_steps_replication_test.go (new):
      • Given a workflow with pre-steps: and a configured safe-outputs.github-app referencing steps.<id>.outputs.*, assert the compiled safe_outputs job contains both the pre-step and the create-github-app-token step, in the correct order.
      • Same for conclusion and activation.
      • Backward-compat: with no scope: specified, default remains agent-only and non-agent jobs are unchanged.
    • Golden-file updates for any affected fixtures under pkg/workflow/testdata/.
    • secret_validation_test.go / security_regression_test.go — confirm strict-mode allowance for secrets.* in replicated steps' with: / env: continues to hold.

Environment

  • gh-aw version: v0.68.4
  • Engine: copilot (also reproduces on claude)
  • Trigger: workflow_dispatch in spike; affects any trigger that causes safe_outputs / conclusion / activation jobs to mint tokens

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions