Skip to content

Aggregate validation errors across workflow compilation passes#13810

Merged
pelikhan merged 5 commits intomainfrom
copilot/implement-error-aggregation-again
Feb 4, 2026
Merged

Aggregate validation errors across workflow compilation passes#13810
pelikhan merged 5 commits intomainfrom
copilot/implement-error-aggregation-again

Conversation

Copy link
Contributor

Copilot AI commented Feb 4, 2026

Validation functions were returning on first error, requiring users to fix issues sequentially. This implements multi-error aggregation using errors.Join() to surface all validation issues in a single compilation pass.

Changes

repository_features_validation.go

  • Aggregate discussions and issues feature checks with ErrorCollector
  • Surface all repository capability errors together

dispatch_workflow_validation.go

  • Collect errors across all workflows in dispatch list
  • Report self-reference, missing files, and trigger issues in one pass

template_validation.go

  • Use errors.Join() for import directives in template regions
  • Aggregate all violations across template blocks

bundler_runtime_validation.go

  • No code changes (already uses fail-fast for recursive dependency resolution)
  • Added comment documenting the fail-fast rationale

Example Output

Before:

Error: dispatch-workflow: self-reference not allowed (workflow 'dispatcher' cannot dispatch itself)

After:

Found 3 dispatch-workflow errors:
  • dispatch-workflow: self-reference not allowed (workflow 'dispatcher' cannot dispatch itself)
  • dispatch-workflow: workflow 'ci' does not support workflow_dispatch trigger
  • dispatch-workflow: workflow 'nonexistent' not found in .github/workflows

All implementations respect the existing --fail-fast flag for users preferring immediate error feedback.

Original prompt

This section details on the original issue you should resolve

<issue_title>[Code Quality] Implement error aggregation for validation using errors.Join()</issue_title>
<issue_description>## Description

Implement multi-error aggregation using errors.Join() (Go 1.20+) in validation files to show users all validation errors at once instead of failing on first error. This dramatically improves feedback loops and developer experience.

Problem

Zero uses of errors.Join() despite Go 1.20+ multi-error support. Validation files return first error and hide subsequent issues. Users must fix errors one-by-one instead of seeing full context, leading to slow feedback loops and frustrated developers.

Suggested Changes

Priority Files to Update

  1. pkg/workflow/bundler_runtime_validation.go
  2. pkg/workflow/repository_features_validation.go
  3. pkg/workflow/dispatch_workflow_validation.go
  4. pkg/workflow/template_validation.go
  5. pkg/workflow/strict_mode_validation.go

Implementation Pattern

func ValidateConfiguration(config *Config) error {
    var errs []error
    
    // Collect all validation errors
    if err := validateField1(config); err != nil {
        errs = append(errs, err)
    }
    
    if err := validateField2(config); err != nil {
        errs = append(errs, err)
    }
    
    if err := validateField3(config); err != nil {
        errs = append(errs, err)
    }
    
    // Return aggregated errors
    if len(errs) > 0 {
        return errors.Join(errs...)
    }
    
    return nil
}

Success Criteria

  • Update validation files to collect all errors before returning
  • Use errors.Join() to aggregate multiple validation errors
  • Maintain error ordering (most critical first)
  • Update tests to verify multi-error behavior
  • Ensure console formatting works with aggregated errors

Testing Example

func TestValidation_MultipleErrors(t *testing.T) {
    config := &Config{
        InvalidField1: "bad",
        InvalidField2: "worse",
    }
    
    err := ValidateConfiguration(config)
    
    require.Error(t, err)
    assert.Contains(t, err.Error(), "invalid field1")
    assert.Contains(t, err.Error(), "invalid field2")
}

Impact

Users see ALL validation issues in one run instead of fixing errors one-by-one.

Source

Extracted from Error Experience Engineering Discussion github/gh-aw#12265

Priority

High - Dramatically improves developer feedback loops

Estimated Effort

Large (1-2 weeks)

AI generated by Discussion Task Miner - Code Quality Improvement Agent

  • expires on Feb 18, 2026, 9:08 PM UTC

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 4, 2026 23:21
- Updated bundler_runtime_validation.go with fail-fast comment (already uses recursive pattern)
- Updated repository_features_validation.go to use ErrorCollector
- Updated dispatch_workflow_validation.go to use ErrorCollector for multiple workflows
- Updated template_validation.go to use errors.Join for template region errors

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Added TestDispatchWorkflowMultipleErrors to verify error aggregation
- Added TestDispatchWorkflowMultipleErrorsFailFast to verify fail-fast mode
- Added TestValidateNoIncludesInTemplateRegions_MultipleErrors for template validation
- Added TestValidateNoIncludesInTemplateRegions_SingleError for single error case
- Fixed missing closing braces in validation functions

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement error aggregation for validation using errors.Join() Aggregate validation errors across workflow compilation passes Feb 4, 2026
Copilot AI requested a review from pelikhan February 4, 2026 23:27
@pelikhan pelikhan marked this pull request as ready for review February 4, 2026 23:41
Copilot AI review requested due to automatic review settings February 4, 2026 23:41
@pelikhan pelikhan merged commit ec588b0 into main Feb 4, 2026
51 checks passed
@pelikhan pelikhan deleted the copilot/implement-error-aggregation-again branch February 4, 2026 23:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements multi-error aggregation for workflow validation functions to improve developer feedback loops. Instead of stopping at the first validation error, the compiler now collects all issues and presents them together using Go's errors.Join() and a new ErrorCollector utility.

Changes:

  • Added error aggregation to dispatch workflow validation, repository features validation, and template validation
  • Implemented ErrorCollector utility to handle fail-fast mode and formatted error output
  • Added comprehensive tests for multi-error scenarios and fail-fast behavior

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
pkg/workflow/template_validation.go Aggregates import directive violations across template blocks
pkg/workflow/template_include_validation_test.go Tests multi-error aggregation and single-error behavior for template validation
pkg/workflow/repository_features_validation.go Collects repository capability errors (discussions/issues) using ErrorCollector
pkg/workflow/dispatch_workflow_validation.go Aggregates errors across workflows in dispatch list (self-references, missing files, trigger issues)
pkg/workflow/dispatch_workflow_test.go Tests multi-error and fail-fast modes for dispatch workflow validation
pkg/workflow/bundler_runtime_validation.go Documentation update explaining fail-fast rationale for recursive dependency resolution

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


if !hasIssues {
return fmt.Errorf("workflow uses safe-outputs.create-issue but repository %s does not have issues enabled. Enable issues in repository settings or remove create-issue from safe-outputs", repo)
// Continue to return aggregated errors even if this check fails
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected spelling of 'to' to 'checking'; the comment should read 'Continue checking other features even if this check fails' to match the pattern established in line 134.

Suggested change
// Continue to return aggregated errors even if this check fails
// Continue checking other features even if this check fails

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(selfRefErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(findErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(notFoundErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(fileReadErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(fileReadErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(compileErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(parseErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(onSectionErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
if returnErr := collector.Add(dispatchErr); returnErr != nil {
return returnErr // Fail-fast mode
}
continue // Skip further validation for this workflow
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple identical comments 'Skip further validation for this workflow' repeated throughout the function create noise. Consider reducing these comments or making them more specific to each validation step (e.g., 'Skip after self-reference check', 'Skip after file read error').

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Code Quality] Implement error aggregation for validation using errors.Join()

2 participants