Skip to content

fix: sync test expectations with COPILOT_API_KEY injection and BYOK default model #28136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 2 commits into from
Apr 23, 2026
+6 −4
Merged

fix: sync test expectations with COPILOT_API_KEY injection and BYOK default model #28136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ceb672a
fix: update test expectations and golden files for COPILOT_API_KEY an…
Copilot Apr 23, 2026
7636908
Merge branch 'main' into copilot/fix-github-actions-workflow-test
pelikhan Apr 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,8 +409,9 @@ jobs:
-- /bin/bash -c 'GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || echo node)"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
env:
COPILOT_AGENT_RUNNER_TYPE: STANDALONE
COPILOT_API_KEY: dummy-byok-key-for-offline-mode
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }}
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || 'claude-sonnet-4.6' }}
Comment on lines 411 to +414
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wasm golden test compiles all fixtures under testdata/wasm_golden/fixtures (including smoke-copilot.md). The corresponding golden output (pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden) still has the old COPILOT_MODEL fallback (|| '') and is missing COPILOT_API_KEY, so TestWasmGolden_CompileFixtures is likely to keep failing. Regenerate/update that golden file as well to reflect the new injected env var and default model fallback.

Copilot uses AI. Check for mistakes.
GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json
GH_AW_PHASE: agent
GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
Expand Down
3 changes: 2 additions & 1 deletion pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,8 +410,9 @@ jobs:
-- /bin/bash -c 'GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || echo node)"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
env:
COPILOT_AGENT_RUNNER_TYPE: STANDALONE
COPILOT_API_KEY: dummy-byok-key-for-offline-mode
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }}
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || 'claude-sonnet-4.6' }}
GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json
GH_AW_PHASE: agent
GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
Expand Down
4 changes: 2 additions & 2 deletions pkg/workflow/threat_detection_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1087,7 +1087,7 @@ func TestCopilotDetectionDefaultModel(t *testing.T) {
shouldContainModel: true,
// Detection uses env var fallback (same pattern as main agent), allowing
// the Copilot CLI to pick its native default (currently claude-sonnet-4.6)
expectedModel: "${{ vars." + constants.EnvVarModelDetectionCopilot + " || '' }}",
expectedModel: "${{ vars." + constants.EnvVarModelDetectionCopilot + " || '" + constants.CopilotBYOKDefaultModel + "' }}",
Comment on lines 1088 to +1090
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment/docs around this assertion still say detection uses an empty-string fallback so the CLI picks its native default, but the expectation was changed to an explicit fallback model (CopilotBYOKDefaultModel). Please update the nearby comment text to match the new behavior (and ideally reference the constant rather than hard-coding a model name) to avoid misleading future readers.

Copilot uses AI. Check for mistakes.
},
{
name: "copilot engine with custom model uses specified model",
Expand Down Expand Up @@ -1133,7 +1133,7 @@ func TestCopilotDetectionDefaultModel(t *testing.T) {
},
},
shouldContainModel: true,
expectedModel: "${{ vars." + constants.EnvVarModelDetectionCopilot + " || '' }}",
expectedModel: "${{ vars." + constants.EnvVarModelDetectionCopilot + " || '" + constants.CopilotBYOKDefaultModel + "' }}",
},
{
name: "claude engine does not add model parameter",
Expand Down
Loading