Skip to content

Add support for pre-agent-steps before agent execution#26666

Merged
pelikhan merged 4 commits intomainfrom
copilot/add-pre-agent-steps-support
Apr 16, 2026
Merged

Add support for pre-agent-steps before agent execution#26666
pelikhan merged 4 commits intomainfrom
copilot/add-pre-agent-steps-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 16, 2026

Summary

  • add new frontmatter field pre-agent-steps
  • support imported pre-agent-steps and merge them with main workflow values
  • inject merged pre-agent-steps immediately before the agent engine execution step
  • apply existing step secret restrictions to pre-agent-steps
  • update schema and frontmatter docs for the new field

Validation

  • go test -v -run "TestPreAgentStepsGeneration|TestPreAgentStepsImportsMergeOrder|TestProcessAndMergePreAgentSteps|TestValidateStepsSecrets" ./pkg/workflow/
  • go test -v ./pkg/parser/...
  • make build
  • make agent-finish (fails due to pre-existing unrelated testifylint issues in pkg/stats/spec_test.go and pkg/testutil/spec_test.go)

Notes

  • parallel_validation code review returned comments for files not changed by this PR; no actionable changes were required in the current diff.
  • CodeQL scan timed out in parallel_validation.

Copilot AI and others added 2 commits April 16, 2026 14:51
Copilot AI requested a review from pelikhan April 16, 2026 15:14
@pelikhan pelikhan marked this pull request as ready for review April 16, 2026 15:48
Copilot AI review requested due to automatic review settings April 16, 2026 15:48
Draft ADR capturing the design decision to introduce `pre-agent-steps`
as a distinct workflow extension point injected immediately before
engine execution.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

Commit pushed: 11534a3

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (pkg/) with 340 new lines across 14 files but does not have a linked Architecture Decision Record (ADR).

AI has analyzed the PR diff and generated a draft ADR to help you get started:

📄 Draft ADR: docs/adr/26666-add-pre-agent-steps-support.md

The draft covers the key design decision to introduce pre-agent-steps as a distinct, named lifecycle extension point injected immediately before engine execution, including alternatives considered (overloading steps, extending pre-steps with phase options, misusing post-steps) and the normative specification for placement, import merge ordering, secret validation, and action pinning.

What to do next

  1. Review the draft ADR committed to your branch — it was generated from the PR diff
  2. Complete any missing sections — add context the AI couldn't infer, refine the decision rationale, and verify that the alternatives listed reflect what you actually considered
  3. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-26666: Introduce pre-agent-steps as a Distinct Workflow Extension Point

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.


📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 26666-add-pre-agent-steps-support.md for PR #26666).

🔒 This PR cannot merge until an ADR is linked in the PR body.

References: §24519930482

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · ● 154.3K ·

@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 81/100

Excellent

Metric Value
New/modified tests analyzed 7 (5 functions + 2 table rows)
✅ Design tests (behavioral contracts) 7 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 5 (71%)
Duplicate test clusters 0
Test inflation detected Yes — 212 test lines vs ~93 production lines (≈2.3:1)
🚨 Coding-guideline violations 1 — missing assertion messages on 5 assert.* calls

Test Classification Details

View all 7 test scenarios
Test File Classification Issues Detected
TestProcessAndMergePreAgentSteps_NoPreAgentSteps compiler_orchestrator_workflow_test.go ✅ Design Missing message on assert.Empty
TestProcessAndMergePreAgentSteps_WithPreAgentSteps compiler_orchestrator_workflow_test.go ✅ Design Happy-path only; missing messages on assert.NotEmpty, assert.Contains
TestProcessAndMergePreAgentSteps_WithImportedPreAgentSteps compiler_orchestrator_workflow_test.go ✅ Design Missing messages on two assert.Contains calls; assert.Less correctly has a message
TestPreAgentStepsGeneration compiler_pre_agent_steps_test.go ✅ Design Full end-to-end ordering contract; uses stdlib assertions (messages not required)
TestPreAgentStepsImportsMergeOrder compiler_pre_agent_steps_test.go ✅ Design Import order contract verified end-to-end
"pre-agent-steps without secrets is allowed" (table row in TestValidateStepsSecrets) strict_mode_steps_validation_test.go ✅ Design Happy-path only
"pre-agent-steps with secret in strict mode fails" (table row in TestValidateStepsSecrets) strict_mode_steps_validation_test.go ✅ Design Error case with errorMsg check

Flagged Tests — Requires Review

⚠️ Missing assertion messages — compiler_orchestrator_workflow_test.go

Five assert.* calls are missing the required descriptive message argument:

// ❌ Missing message — hard to diagnose failures
assert.Empty(t, workflowData.PreAgentSteps)
assert.NotEmpty(t, workflowData.PreAgentSteps)
assert.Contains(t, workflowData.PreAgentSteps, "Prepare final context")
assert.Contains(t, workflowData.PreAgentSteps, "Main pre-agent step")
assert.Contains(t, workflowData.PreAgentSteps, "Imported pre-agent step")

// ✅ Correct — has message
require.NoError(t, err, "yaml.Marshal should not fail for well-formed pre-agent-steps")
assert.Less(t, importedIdx, mainIdx, "Imported pre-agent-steps should come before main pre-agent-steps")

Suggested fix — add descriptive messages to every assert.* call:

assert.Empty(t, workflowData.PreAgentSteps, "PreAgentSteps should be empty when no pre-agent-steps defined")
assert.NotEmpty(t, workflowData.PreAgentSteps, "PreAgentSteps should be populated from frontmatter")
assert.Contains(t, workflowData.PreAgentSteps, "Prepare final context", "PreAgentSteps should include step name from frontmatter")
assert.Contains(t, workflowData.PreAgentSteps, "Main pre-agent step", "PreAgentSteps should contain main workflow step")
assert.Contains(t, workflowData.PreAgentSteps, "Imported pre-agent step", "PreAgentSteps should contain imported step")

i️ Test inflation — compiler_pre_agent_steps_test.go

compiler_pre_agent_steps_test.go adds 126 lines against its closest production counterpart workflow_builder.go (+54 lines), a ratio of ≈ 2.3:1 (threshold: 2:1). Overall test:production ratio is 212:93 (≈ 2.3:1). This is marginal — the two end-to-end compilation tests (TestPreAgentStepsGeneration, TestPreAgentStepsImportsMergeOrder) necessarily include scaffolding to write temp files, compile, and parse output, which inflates line count without inflating test count. The verbosity is justified here, but be aware of future growth.

i️ Happy-path gap — TestProcessAndMergePreAgentSteps_WithPreAgentSteps

This test only covers the successful case (steps are populated). Consider adding a complementary sub-case or table row that verifies the function behaves correctly when the pre-agent-steps key contains an empty slice []any{}.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 7 test scenarios — all unit (//go:build !integration), no missing build tags

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All tests verify behavioral contracts. One guideline nit: add descriptive messages to 5 bare assert.* calls in compiler_orchestrator_workflow_test.go.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References:

🧪 Test quality analysis by Test Quality Sentinel · ● 686.2K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 81/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 7 new test scenarios verify behavioral contracts. Minor nit: add descriptive messages to 5 bare assert.* calls in compiler_orchestrator_workflow_test.go.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new pre-agent-steps frontmatter field that can be imported/merged and is injected immediately before the agent engine execution step, with the same strict-mode secret-expression restrictions as other custom step sections.

Changes:

  • Add pre-agent-steps to workflow frontmatter types/serialization, schema validation, and user docs.
  • Support importing/merging pre-agent-steps from imported workflows and apply action pinning during merge.
  • Emit merged pre-agent-steps in the generated main job YAML right before engine execution; add unit tests covering generation, import merge order, and strict-mode secret validation.
Show a summary per file
File Description
pkg/workflow/workflow_builder.go Adds merge logic for imported + main pre-agent-steps, with action pinning and YAML serialization into WorkflowData.
pkg/workflow/strict_mode_steps_validation_test.go Extends strict-mode secret validation tests to cover pre-agent-steps.
pkg/workflow/strict_mode_steps_validation.go Includes pre-agent-steps in strict-mode step secret scanning.
pkg/workflow/frontmatter_types.go Adds PreAgentSteps to the parsed frontmatter config struct.
pkg/workflow/frontmatter_serialization.go Serializes pre-agent-steps when converting FrontmatterConfig to a map.
pkg/workflow/compiler_yaml_main_job.go Injects pre-agent-steps right before engine execution in the main job.
pkg/workflow/compiler_yaml.go Adds generatePreAgentSteps to emit the section via the shared steps writer.
pkg/workflow/compiler_types.go Adds PreAgentSteps to WorkflowData.
pkg/workflow/compiler_pre_agent_steps_test.go New tests asserting correct placement and import merge ordering in generated lock workflow YAML.
pkg/workflow/compiler_orchestrator_workflow_test.go Adds unit tests for processAndMergePreAgentSteps behavior.
pkg/workflow/compiler_orchestrator_workflow.go Wires processAndMergePreAgentSteps into workflow parsing.
pkg/parser/schemas/main_workflow_schema.json Adds schema definition for pre-agent-steps.
pkg/parser/import_processor.go Extends ImportsResult with MergedPreAgentSteps.
pkg/parser/import_field_extractor.go Extracts pre-agent-steps from imports and accumulates them for merging.
docs/src/content/docs/reference/frontmatter.md Documents the new pre-agent-steps field.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 16/16 changed files
  • Comments generated: 3

Comment on lines +3 to +4
// It validates that secrets expressions are not used in custom steps (steps,
// pre-agent-steps, and post-steps injected in the agent job). In strict mode, secrets in step-level
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File header comment says strict mode validation applies to steps/pre-agent-steps/post-steps, but validateStepsSecrets also validates the pre-steps section. Update the header comment to include pre-steps so it matches actual behavior.

Suggested change
// It validates that secrets expressions are not used in custom steps (steps,
// pre-agent-steps, and post-steps injected in the agent job). In strict mode, secrets in step-level
// It validates that secrets expressions are not used in custom steps (pre-steps,
// steps, pre-agent-steps, and post-steps injected in the agent job). In strict mode, secrets in step-level

Copilot uses AI. Check for mistakes.
Comment thread pkg/workflow/compiler_yaml.go Outdated
Comment on lines 652 to 654
// writeStepsSection writes a steps section (pre-steps or post-steps) to the YAML builder,
// stripping the header line and normalising indentation to match the agent job step format:
// top-level items get 6-space indent ( - name:) and nested properties get 8-space indent ( run:).
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeStepsSection is now used for pre-agent-steps as well (via generatePreAgentSteps), but the comment still says it only writes pre-steps/post-steps. Please update the comment to include pre-agent-steps so future readers don’t miss this usage.

See below for a potential fix:

// writeStepsSection writes a steps section (pre-steps, post-steps, or pre-agent-steps)
// to the YAML builder, stripping the header line and normalising indentation to match
// the agent job step format: top-level items get 6-space indent (      - name:) and
// nested properties get 8-space indent (        run:).
func writeStepsSection(yaml *strings.Builder, stepsYAML string) {
	if stepsYAML == "" {
		return
	}
	lines := strings.Split(stepsYAML, "\n")
	for _, line := range lines[1:] { // skip the "pre-steps:" / "post-steps:" / "pre-agent-steps:" header line

Copilot uses AI. Check for mistakes.
```

Use pre-agent steps when work must happen right before the engine runs (for example, final context preparation or last-moment validations).

Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new pre-agent-steps section doesn’t mention that these steps execute outside the firewall sandbox (like steps: and post-steps: do). Consider adding a sentence clarifying the security/runtime context for pre-agent-steps for consistency and to avoid confusion.

Suggested change
Pre-agent steps run outside the firewall sandbox. These steps execute with standard GitHub Actions security.

Copilot uses AI. Check for mistakes.
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 16, 2026

@copilot review all comments

Addressed all actionable review comments in commit f1dc82f: updated strict-mode header wording to include pre-steps, updated writeStepsSection comment to include pre-agent-steps, and added docs clarification that pre-agent-steps run outside the firewall sandbox.

@pelikhan pelikhan merged commit 25dc149 into main Apr 16, 2026
@pelikhan pelikhan deleted the copilot/add-pre-agent-steps-support branch April 16, 2026 16:08
Copilot AI restored the copilot/add-pre-agent-steps-support branch April 16, 2026 16:09
Copilot stopped work on behalf of pelikhan due to an error April 16, 2026 16:09
@github-actions github-actions Bot deleted the copilot/add-pre-agent-steps-support branch April 24, 2026 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants