Skip to content

Fix missing awmg integration test coverage and optimize CI matrix#8521

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-awmg-integration-test-coverage
Jan 1, 2026
Merged

Fix missing awmg integration test coverage and optimize CI matrix#8521
pelikhan merged 2 commits intomainfrom
copilot/fix-awmg-integration-test-coverage

Conversation

Copy link
Contributor

Copilot AI commented Jan 1, 2026

The awmg package integration tests were not executing in CI due to compilation errors and missing matrix configuration. This caused 3 test files with MCP gateway tests to be completely untested in production.

Changes

CI Matrix Optimization (.github/workflows/ci.yml)

  • Added "AWMG Gateway Tests" matrix entry for ./pkg/awmg package
  • Removed "Workflow Expression & Safety" and "Workflow Job Management" groups (0 tests matched)
  • Updated skip patterns in catch-all groups
  • Documented slow tests: CLI Progress Flag (~65s), CLI HTTP MCP Connect (~43s)
  • Net result: 33 → 32 matrix groups

Fixed awmg Test Compilation (pkg/awmg/*_test.go)

  • Fixed struct literal errors when initializing parser.MCPServerConfig
  • Added types import to gateway test files
  • Updated 8 struct initializations across 3 files

The issue was Go's limitation with embedded struct fields in literals:

// Before (compilation error)
config := parser.MCPServerConfig{
    Command: binaryPath,
    Args:    []string{"mcp-server"},
}

// After (correct)
config := parser.MCPServerConfig{
    BaseMCPServerConfig: types.BaseMCPServerConfig{
        Command: binaryPath,
        Args:    []string{"mcp-server"},
    },
}

All 21 awmg integration tests now compile and pass (22.87s locally).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/user
    • Triggering command: /usr/bin/gh gh api user --jq .login ut-3197140303.c .cfg 0.1-go1.25.0.lin/var/log/sysstat --gdwarf-5 --64 (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[ci-coach] Fix missing awmg integration test coverage and optimize CI matrix</issue_title>
<issue_description>## CI Optimization - Critical Coverage Gap + Matrix Cleanup

This PR addresses a critical test coverage gap discovered during automated CI analysis, along with efficiency improvements to the integration test matrix.


🚨 Critical Finding: Missing Test Coverage

Issue: Integration tests for ./pkg/awmg package were NOT being executed in CI

Impact:

  • 3 integration test files with MCP gateway tests were completely untested
  • Production code may have regressions in MCP gateway functionality

Files affected:

  • ./pkg/awmg/gateway_integration_test.go
  • ./pkg/awmg/gateway_streamable_http_test.go
  • ./pkg/awmg/gateway_inspect_integration_test.go

Fix: ✅ Added matrix entry "AWMG Gateway Tests" to execute all awmg integration tests


🧹 Optimization: Matrix Cleanup

Issue: Empty test groups wasting CI slots

Groups removed (0 tests each):

  • "Workflow Expression & Safety" - pattern matched no existing tests
  • "Workflow Job Management" - pattern matched no existing tests

Fix: ✅ Removed 2 empty groups from integration matrix


📊 Matrix Balance Analysis

Current state after optimizations:

  • Total matrix groups: 32 (was 33)
    • Removed 2 empty groups
    • Added 1 new group (awmg)
  • Coverage improvement: 100% of awmg tests now executed
  • Time impact: +3-5 seconds for new awmg tests (acceptable for critical coverage)

Notable slow groups (documented for future optimization):

  1. CLI Progress Flag: ~65s (TestProgressFlagSignature)
  2. CLI HTTP MCP Connect: ~43s (TestConnectHTTPMCPServer)

Added inline comments in CI config to document these slow tests for future performance work.


✅ Changes Made

.github/workflows/ci.yml

  • Line 169-171: Added "AWMG Gateway Tests" matrix entry for ./pkg/awmg package
  • Line 87-89: Added comment noting CLI Progress Flag slow test (~65s)
  • Line 91-93: Added comment noting CLI HTTP MCP Connect slow tests (~43s)
  • Removed: "Workflow Expression & Safety" group (no matching tests)
  • Removed: "Workflow Job Management" group (no matching tests)
  • Updated: Skip pattern in "Workflow Misc Part 2" catch-all to reflect removed groups

📈 Impact Summary

Metric Before After Change
Matrix groups 33 32 -1 (net)
Empty groups 2 0 -2 ✅
Packages covered 4 5 +1 ✅
AWMG test coverage 0% 100% +100% 🎯
Time per run baseline +3-5s Small cost for critical coverage

🔍 Analysis Methodology

This optimization was discovered through:

  1. Test coverage analysis: Scanned all integration test files vs. CI matrix packages
  2. Timing analysis: Analyzed last 100 CI runs and integration test artifacts
  3. Pattern matching: Identified groups with no matching tests (0.0s duration)
  4. Gap detection: Found ./pkg/awmg package with integration tests but no CI coverage

Data analyzed:

  • 100 recent CI workflow runs
  • 714 total test files (43 integration, 671 unit)
  • 33 integration matrix groups (timing data from artifacts)

🎯 Risk Assessment

Risk Level: Very Low

Added coverage is critical - MCP gateway tests were previously untested
Removed groups are safe - confirmed 0 tests matched their patterns
No test logic changes - only matrix grouping modified
Documented slow tests - for future optimization without changing behavior


🧪 Testing Plan

  • Monitor first CI run to verify awmg tests execute correctly
  • Confirm no tests are missed by empty group removal
  • Validate new awmg group completes in reasonable time (<10s expected)
  • Check that slow test comments are visible in GitHub Actions UI

📚 Related Work


🔮 Future Optimization Opportunities

Documented but not addressed in this PR:

  1. TestProgressFlagSignature (~65s) - Consider mocking or optimizing
  2. TestConnectHTTPMCPServer (~43s) - Consider splitting into smaller cases
  3. Matrix rebalancing - Could split slow groups for better parallelization

These are marked with inline comments in the CI config for future work.


Metrics Baseline (for future comparison):

  • Total CI runs analyzed: 100
  • Success rate: 35%
  • Average integration group time: 10.8s
  • Longest group: CLI Progress Flag (65.3s)

References:

AI generated by [CI Optimization Coach](https://github.com/githubnext/gh-aw/a...


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Add AWMG Gateway Tests matrix entry for ./pkg/awmg package
- Remove "Workflow Expression & Safety" group (0 tests matched)
- Remove "Workflow Job Management" group (0 tests matched)
- Add comments documenting slow CLI tests (~65s and ~43s)
- Update skip_pattern in Workflow Misc Part 2 to reflect removed groups
- Fix awmg integration test compilation errors:
  - Add types import to gateway_integration_test.go
  - Add types import to gateway_inspect_integration_test.go
  - Update MCPServerConfig struct initialization to use BaseMCPServerConfig
  - Fix all 8 struct literal compilation errors across 3 test files

Matrix changes: 33 → 32 groups (net -1, removed 2 + added 1)
Critical fix: awmg integration tests now compile and can run in CI

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix missing AWMG integration test coverage and optimize CI matrix Fix missing awmg integration test coverage and optimize CI matrix Jan 1, 2026
Copilot AI requested a review from pelikhan January 1, 2026 22:51
@pelikhan pelikhan marked this pull request as ready for review January 1, 2026 22:53
@pelikhan pelikhan merged commit 7d9d17b into main Jan 1, 2026
169 checks passed
@pelikhan pelikhan deleted the copilot/fix-awmg-integration-test-coverage branch January 1, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ci-coach] Fix missing awmg integration test coverage and optimize CI matrix

2 participants