GHES: Replace opaque Copilot errors with actionable messages#20976
GHES: Replace opaque Copilot errors with actionable messages#20976
Conversation
- Create detect_ghes_copilot_errors.sh script to detect common GHES errors - Add actionable error message templates for each GHES error type - Integrate GHES error detection into Copilot engine workflow generation - Add error outputs to agent job and pass to conclusion job - Update failure message templates to include GHES error context - Support detection of: token exchange 403, model loading 400, firewall blocks, gh CLI misconfiguration Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
- Add comprehensive tests for GHES error detection in workflow compilation - Update golden test files to include new GHES error outputs - Verify tests pass for Copilot engine workflows Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
|
@claude[agent] fix the pr title and summary |
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Updated PR title to "GHES: Replace opaque Copilot errors with actionable messages" and added comprehensive description with implementation details, testing information, and example error messages. (8a960c1) Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
This PR adds comprehensive GHES-specific error detection for the Copilot engine, replacing opaque error messages with clear, actionable guidance for common GHES deployment issues. ChangesDetection Script
|
There was a problem hiding this comment.
Pull request overview
Adds GHES-specific GitHub Copilot error detection to generated workflows and surfaces those signals in the agent-failure issue/comment templates so failures on GitHub Enterprise Server can be diagnosed with clearer remediation guidance.
Changes:
- Add a new post-run agent step (
detect-ghes-errors) plus job outputs for several GHES Copilot failure modes (token exchange 403, model loading 400, firewall block, gh CLI misconfig). - Plumb the new outputs into the conclusion job env and extend the agent-failure handler to render GHES-specific failure context from new prompt templates.
- Add a new shell detector script, new markdown prompt templates, golden/lock workflow regenerations, and compiler tests.
Reviewed changes
Copilot reviewed 131 out of 131 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/copilot_engine_execution.go | Introduces generator for the GHES error detection step. |
| pkg/workflow/compiler_yaml_main_job.go | Emits the GHES detection step into Copilot engine main job steps. |
| pkg/workflow/compiler_main_job.go | Exposes GHES error flags as main-job outputs for Copilot workflows. |
| pkg/workflow/notify_comment.go | Passes GHES error outputs into conclusion job env for failure reporting. |
| pkg/workflow/ghes_copilot_errors_test.go | Adds compiler tests asserting step/output/env plumbing for Copilot vs non-Copilot engines. |
| actions/setup/sh/detect_ghes_copilot_errors.sh | Adds log-scraping script that sets GHES-specific error outputs for GitHub Actions. |
| actions/setup/js/handle_agent_failure.cjs | Renders GHES-specific failure context into agent-failure issues/comments using new prompt templates. |
| actions/setup/md/ghes_token_exchange_error.md | New prompt: GHES Copilot token exchange 403 guidance. |
| actions/setup/md/ghes_model_loading_error.md | New prompt: GHES Copilot model loading 400 guidance. |
| actions/setup/md/ghes_firewall_block_error.md | New prompt: firewall block guidance for GHES API domain access. |
| actions/setup/md/ghes_gh_cli_error.md | New prompt: gh CLI misconfiguration guidance for GHES. |
| actions/setup/md/agent_failure_issue.md | Adds {ghes_error_context} placeholder to failure issue template. |
| actions/setup/md/agent_failure_comment.md | Adds {ghes_error_context} placeholder to failure comment template. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/basic-copilot.golden | Updates golden output to include GHES detection step + outputs. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden | Updates golden output to include GHES detection step + outputs. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-imports.golden | Updates golden output to include GHES detection step + outputs. |
| .github/workflows/workflow-skill-extractor.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/workflow-normalizer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/workflow-health-manager.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/workflow-generator.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/weekly-safe-outputs-spec-review.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/weekly-issue-summary.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/weekly-editors-health-check.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/video-analyzer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/ubuntu-image-analyzer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/tidy.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/test-workflow.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
| .github/workflows/test-project-url-default.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/test-dispatcher.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/terminal-stylist.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/technical-doc-writer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/super-linter.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/sub-issue-closer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/stale-repo-identifier.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-workflow-call.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-workflow-call-with-inputs.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-update-cross-repo-pr.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-test-tools.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-temporary-id.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-project.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-multi-pr.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-create-cross-repo-pr.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-copilot.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/smoke-copilot-arm.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/slide-deck-maintainer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/security-review.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/security-compliance.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/research.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/repository-quality-improver.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/repo-tree-map.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/repo-audit-analyzer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/release.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/refiner.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/q.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/python-data-charts.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/pr-triage-agent.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/pr-nitpick-reviewer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/portfolio-analyst.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/poem-bot.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/plan.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/pdf-summary.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/org-health-report.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/notion-issue-summary.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/metrics-collector.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
| .github/workflows/mergefest.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/mcp-inspector.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/layout-spec-maintainer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/jsweep.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/issue-triage-agent.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/issue-monster.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/hourly-ci-cleaner.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/grumpy-reviewer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/gpclean.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/glossary-maintainer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/github-remote-mcp-auth-test.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/functional-pragmatist.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/firewall.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
| .github/workflows/firewall-escape.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/example-permissions-warning.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
| .github/workflows/draft-pr-cleanup.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/docs-noob-tester.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/discussion-task-miner.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dictation-prompt.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dev.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dev-hawk.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dependabot-go-checker.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dependabot-burner.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/delight.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/dead-code-remover.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-workflow-updater.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-testify-uber-super-expert.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-team-status.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-syntax-error-quality.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-semgrep-scan.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-secrets-analysis.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-repo-chronicle.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-regulatory.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-performance-summary.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-news.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-mcp-concurrency-analysis.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-malicious-code-scan.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-firewall-report.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-file-diet.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-copilot-token-report.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-compiler-quality.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-cli-tools-tester.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-cli-performance.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-assign-issue-to-user.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/daily-architecture-diagram.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/copilot-pr-prompt-analysis.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/copilot-pr-nlp-analysis.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/copilot-pr-merged-report.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/copilot-cli-deep-research.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/constraint-solving-potd.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/code-simplifier.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/code-scanning-fixer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/cli-consistency-checker.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/ci-doctor.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/ci-coach.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/chroma-issue-indexer.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
| .github/workflows/breaking-change-checker.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/brave.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/bot-detection.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/auto-triage-issues.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/artifacts-summary.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/archie.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/agent-persona-explorer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/agent-performance-analyzer.lock.yml | Regenerated lock workflow to include GHES detection step, outputs, and conclusion env vars. |
| .github/workflows/ace-editor.lock.yml | Regenerated lock workflow to include GHES detection step and outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| # Check for gh CLI misconfiguration (targeting github.com instead of GHES) | ||
| # Pattern: "Could not resolve to a Repository" or "gh.*github.com" when GHES is expected | ||
| if grep -qE "(Could not resolve to a Repository|gh.*targeting.*github\.com|GH_HOST.*not set)" "$LOG_FILE"; then | ||
| echo "Detected gh CLI misconfiguration (targeting github.com instead of GHES)" | ||
| echo "ghes_gh_cli_misconfigured=true" >> "$GITHUB_OUTPUT" |
Summary
This PR adds comprehensive GHES-specific error detection for the Copilot engine, replacing opaque error messages with clear, actionable guidance for common GHES deployment issues.
Changes
Detection Script
detect_ghes_copilot_errors.shto detect 4 common GHES error patterns:Error Message Templates
ghes_token_exchange_error.md- Licensing and GitHub Connect resolution stepsghes_model_loading_error.md- API proxy troubleshootingghes_firewall_block_error.md- Firewall configuration guidanceghes_gh_cli_error.md- gh CLI configuration fixesCompiler Integration
ghes_token_exchange_403,ghes_model_loading_400,ghes_firewall_block,ghes_gh_cli_misconfigured)Tests
ghes_copilot_errors_test.go:Testing
All tests pass:
Example Error Messages
When a GHES-specific error is detected, users will see actionable guidance like:
Token Exchange 403:
Model Loading 400:
Firewall Block:
gh CLI Misconfiguration: