Skip to content

Fix JavaScript test failures: schema validation and missing template files#10125

Merged
pelikhan merged 3 commits intomainfrom
copilot/fix-js-test-suite-failures
Jan 15, 2026
Merged

Fix JavaScript test failures: schema validation and missing template files#10125
pelikhan merged 3 commits intomainfrom
copilot/fix-js-test-suite-failures

Conversation

Copy link
Contributor

Copilot AI commented Jan 15, 2026

CI run #21041144963 reported 15 JavaScript test failures across two test files. Root causes: missing examples in safe-output tool schemas and absent template files in test environment.

Changes

Schema Documentation (safe_outputs_tools.json)

  • Added concrete examples to autofix_code_scanning_alert fields:
    "fix_description": "... Example: 'Sanitize user input to prevent SQL injection by using parameterized queries...'"
    "fix_code": "... Example for SQL injection fix: 'query = db.prepare(\"SELECT * FROM users WHERE id = ?\").bind(userId)'"
  • Added use-case language to create_project_status_update: "Use this when you need to provide stakeholder updates..."

Test Environment Setup

  • Created /opt/gh-aw/prompts/ directory and populated with template files from actions/setup/md/
  • Matches CI workflow behavior from commit c07e9a4
  • Enables handle_agent_failure.cjs tests to execute locally

Result

All 2697 JavaScript tests pass (previously 15 failures).

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] JavaScript test suite failure: 44 tests failing due to incomplete safe-output tool schema sync</issue_title>
<issue_description># 🏥 CI Failure Investigation - Run githubnext/gh-aw#29790

Summary

CI failed with 44 JavaScript test failures across 4 test files after attempting to sync missing safe output tool definitions.

Failure Details

  • Run: #21041144963
  • Commit: 35fe12f0c8ae50dae4b3fba9da525d23174f63e9
  • Trigger: Push to main
  • Failed Job: js
  • Failed Step: Run tests
  • Severity: CRITICAL

Root Cause

Incomplete schema synchronization between:

  1. Tool definitions (JSON)
  2. Validation configuration (Go)
  3. Implementation (JavaScript)
  4. Tests

Test Failure Breakdown

  1. 36 failures in collect_ndjson_output.test.cjs - Validation logic broken
  2. 4 failures in mcp_handler_go.test.cjs - Go binary not found
  3. 2 failures in safe_outputs_mcp_schema_validation.test.cjs - Missing examples
  4. 2 failures in safe_outputs_mcp_server_defaults.test.cjs - Tests timeout

Key Problems

1. Validation Not Working

Tests expect validation errors but get none:

expected [] to have a length of 1 but got +0

2. Incomplete Schema Documentation

  • autofix_code_scanning_alert: Missing examples for fix_description and fix_code
  • create_project_status_update: Missing use case description

3. MCP Server Issues

Branch parameter tests timing out, suggesting configuration problems.

Recommended Actions

Option 1: Revert

git revert 35fe12f0c8ae50dae4b3fba9da525d23174f63e9

Option 2: Fix Forward

  1. Complete schema documentation
  2. Fix validation logic
  3. Fix MCP server configuration
  4. Verify: cd actions/setup/js && npm test

Files to Investigate

  • pkg/workflow/js/safe_outputs_tools.json
  • pkg/workflow/safe_output_validation_config.go
  • actions/setup/js/collect_ndjson_output.cjs
  • actions/setup/js/safe_outputs_mcp_server.cjs

Prevention

When modifying safe-output schemas, update ALL files together:

  1. Tool schema JSON
  2. Validation config Go
  3. Implementation JavaScript
  4. Tests
  5. Go structs

Always run before commit:

cd actions/setup/js && npm test
cd ../../.. && make test
make agent-finish

Related Issues

Historical Context

Second schema sync issue in 24 hours, indicating the sync process needs improvement.


AI generated by CI Failure Doctor

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Add examples to autofix_code_scanning_alert fix_description and fix_code fields
- Update create_project_status_update description with clear use case language
- All schema validation tests now pass

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix JavaScript test suite failures due to schema sync issues Fix JavaScript test failures: schema validation and missing template files Jan 15, 2026
Copilot AI requested a review from mnkiefer January 15, 2026 19:31
@pelikhan pelikhan marked this pull request as ready for review January 15, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] JavaScript test suite failure: 44 tests failing due to incomplete safe-output tool schema sync

3 participants