fix(test): Windows Unicode encoding in integration tests#296
Merged
danielmeppiel merged 2 commits intomainfrom Mar 14, 2026
Merged
fix(test): Windows Unicode encoding in integration tests#296danielmeppiel merged 2 commits intomainfrom
danielmeppiel merged 2 commits intomainfrom
Conversation
- Replace Unicode symbols (✓, ✨) with ASCII equivalents in test output to avoid UnicodeEncodeError on Windows cp1252 stdout - Add encoding='utf-8' and errors='replace' to subprocess.run/Popen calls so APM's Rich-formatted output can be captured on Windows - Applied to guardrailing, MCP registry, and ADO test files Fixes the second Windows CI failure (after teardown fix in #295): UnicodeEncodeError: 'charmap' codec can't encode character '\u2713' Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes Windows integration test failures caused by non-UTF-8 console/codepage defaults by making test output ASCII-safe and forcing UTF-8 decoding for captured subprocess output.
Changes:
- Replace Unicode status symbols in E2E test
print()statements with ASCII equivalents. - Add
encoding='utf-8'anderrors='replace'tosubprocess.run(..., text=True)calls that capture APM output. - Apply the same subprocess decoding hardening to additional E2E suites (preemptive).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/integration/test_guardrailing_hero_e2e.py |
Forces UTF-8 decoding for subprocess output and replaces Unicode symbols in status prints. |
tests/integration/test_mcp_registry_e2e.py |
Forces UTF-8 decoding for subprocess output and replaces most Unicode status prints with ASCII markers. |
tests/integration/test_ado_e2e.py |
Forces UTF-8 decoding for captured subprocess output. |
Comments suppressed due to low confidence (1)
tests/integration/test_mcp_registry_e2e.py:417
- There is still a non-ASCII warning symbol printed in this flow ("⚠ Copilot configuration not created..."). On Windows with a non-UTF-8 console encoding this can still raise
UnicodeEncodeError, defeating the goal of replacing unicode symbols in test output. Replace it with an ASCII-only marker (e.g., "[WARN]") to keep this suite Windows-safe.
print(f"[OK] Copilot configuration tested with {len(mcp_servers)} servers")
except json.JSONDecodeError as e:
pytest.fail(f"Invalid JSON in Copilot config: {e}\nContent: {config_content}")
else:
Comment on lines
41
to
68
| if show_output: | ||
| print(f"\n>>> Running command: {cmd}") | ||
| result = subprocess.run( | ||
| cmd, | ||
| shell=True, | ||
| check=check, | ||
| capture_output=False, | ||
| text=True, | ||
| timeout=timeout, | ||
| cwd=cwd, | ||
| env=env | ||
| env=env, | ||
| encoding='utf-8', | ||
| errors='replace' | ||
| ) | ||
| result_capture = subprocess.run( | ||
| cmd, | ||
| shell=True, | ||
| check=False, | ||
| capture_output=True, | ||
| text=True, | ||
| timeout=timeout, | ||
| cwd=cwd, | ||
| env=env | ||
| env=env, | ||
| encoding='utf-8', | ||
| errors='replace' | ||
| ) | ||
| result.stdout = result_capture.stdout | ||
| result.stderr = result_capture.stderr |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens Windows integration/E2E tests against Unicode encoding/decoding issues by switching test log markers to ASCII and forcing UTF-8 decoding for subprocess output that captures APM CLI output.
Changes:
- Replace Unicode markers in test
print()statements (e.g.,✓,✨) with ASCII equivalents (e.g.,[OK]). - Add
encoding='utf-8'anderrors='replace'tosubprocess.run(...)/Popen(...)calls that capture text output.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tests/integration/test_guardrailing_hero_e2e.py |
Uses UTF-8 decoding for subprocess output and replaces Unicode log markers with ASCII. |
tests/integration/test_mcp_registry_e2e.py |
Uses UTF-8 decoding for subprocess output and replaces Unicode log markers with ASCII. |
tests/integration/test_ado_e2e.py |
Uses UTF-8 decoding for captured subprocess output (preemptive Windows hardening). |
Comments suppressed due to low confidence (1)
tests/integration/test_mcp_registry_e2e.py:154
- These logs still print
result.stdoutdirectly.apm mcp search/showuses RichTable(...)output (box-drawing characters) and other commands emit Unicode symbols (e.g., "✓" in install output). On Windows with a cp1252 stdout, printing that captured text can still raiseUnicodeEncodeErroreven after replacing the leading test marker. Consider sanitizing before printing (encode to the current stdout encoding witherrors='replace'/backslashreplace), or avoid printing captured stdout in the test output.
print(f"[OK] MCP search found GitHub servers:\n{result.stdout[:500]}...")
| assert "installation" in output or "install" in output or "command" in output, "Should show installation info" | ||
|
|
||
| print(f"✓ MCP show displays server details:\n{result.stdout[:500]}...") | ||
| print(f"[OK] MCP show displays server details:\n{result.stdout[:500]}...") |
Comment on lines
205
to
+209
| text=True, | ||
| cwd=project_dir, | ||
| env=env | ||
| env=env, | ||
| encoding='utf-8', | ||
| errors='replace' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After fixing the teardown issue (PR #295), the Windows release pipeline progressed to the next test suite (guardrailing hero scenario) and hit a new failure:
Windows uses cp1252 encoding for stdout by default, which cannot encode Unicode symbols like
✓and✨. Additionally,subprocess.run(text=True)defaults to cp1252 for decoding APM output, which contains Rich-formatted Unicode.Fix
✓,✨) with ASCII equivalents ([OK], etc.) in test print statementsencoding='utf-8'+errors='replace'to all subprocess calls that capture APM output — ensures Rich's Unicode output can be decoded on WindowsFiles changed
tests/integration/test_guardrailing_hero_e2e.py— print statements + subprocess encodingtests/integration/test_mcp_registry_e2e.py— print statements + subprocess encoding (preemptive — next suite after guardrailing)tests/integration/test_ado_e2e.py— subprocess encoding (preemptive)Context