Skip to content

fix(test): Windows Unicode encoding in integration tests#296

Merged
danielmeppiel merged 2 commits intomainfrom
fix/windows-unicode-integration-tests
Mar 14, 2026
Merged

fix(test): Windows Unicode encoding in integration tests#296
danielmeppiel merged 2 commits intomainfrom
fix/windows-unicode-integration-tests

Conversation

@danielmeppiel
Copy link
Collaborator

Problem

After fixing the teardown issue (PR #295), the Windows release pipeline progressed to the next test suite (guardrailing hero scenario) and hit a new failure:

UnicodeEncodeError: 'charmap' codec can't encode character '\u2713' in position 0

Windows uses cp1252 encoding for stdout by default, which cannot encode Unicode symbols like and . Additionally, subprocess.run(text=True) defaults to cp1252 for decoding APM output, which contains Rich-formatted Unicode.

Fix

  1. Replaced Unicode symbols (, ) with ASCII equivalents ([OK], etc.) in test print statements
  2. Added encoding='utf-8' + errors='replace' to all subprocess calls that capture APM output — ensures Rich's Unicode output can be decoded on Windows

Files changed

  • tests/integration/test_guardrailing_hero_e2e.py — print statements + subprocess encoding
  • tests/integration/test_mcp_registry_e2e.py — print statements + subprocess encoding (preemptive — next suite after guardrailing)
  • tests/integration/test_ado_e2e.py — subprocess encoding (preemptive)

Context

- Replace Unicode symbols (✓, ✨) with ASCII equivalents in test output
  to avoid UnicodeEncodeError on Windows cp1252 stdout
- Add encoding='utf-8' and errors='replace' to subprocess.run/Popen calls
  so APM's Rich-formatted output can be captured on Windows
- Applied to guardrailing, MCP registry, and ADO test files

Fixes the second Windows CI failure (after teardown fix in #295):
  UnicodeEncodeError: 'charmap' codec can't encode character '\u2713'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 14, 2026 11:43
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Windows integration test failures caused by non-UTF-8 console/codepage defaults by making test output ASCII-safe and forcing UTF-8 decoding for captured subprocess output.

Changes:

  • Replace Unicode status symbols in E2E test print() statements with ASCII equivalents.
  • Add encoding='utf-8' and errors='replace' to subprocess.run(..., text=True) calls that capture APM output.
  • Apply the same subprocess decoding hardening to additional E2E suites (preemptive).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/integration/test_guardrailing_hero_e2e.py Forces UTF-8 decoding for subprocess output and replaces Unicode symbols in status prints.
tests/integration/test_mcp_registry_e2e.py Forces UTF-8 decoding for subprocess output and replaces most Unicode status prints with ASCII markers.
tests/integration/test_ado_e2e.py Forces UTF-8 decoding for captured subprocess output.
Comments suppressed due to low confidence (1)

tests/integration/test_mcp_registry_e2e.py:417

  • There is still a non-ASCII warning symbol printed in this flow ("⚠ Copilot configuration not created..."). On Windows with a non-UTF-8 console encoding this can still raise UnicodeEncodeError, defeating the goal of replacing unicode symbols in test output. Replace it with an ASCII-only marker (e.g., "[WARN]") to keep this suite Windows-safe.
                    print(f"[OK] Copilot configuration tested with {len(mcp_servers)} servers")
                    
                except json.JSONDecodeError as e:
                    pytest.fail(f"Invalid JSON in Copilot config: {e}\nContent: {config_content}")
            else:

Comment on lines 41 to 68
if show_output:
print(f"\n>>> Running command: {cmd}")
result = subprocess.run(
cmd,
shell=True,
check=check,
capture_output=False,
text=True,
timeout=timeout,
cwd=cwd,
env=env
env=env,
encoding='utf-8',
errors='replace'
)
result_capture = subprocess.run(
cmd,
shell=True,
check=False,
capture_output=True,
text=True,
timeout=timeout,
cwd=cwd,
env=env
env=env,
encoding='utf-8',
errors='replace'
)
result.stdout = result_capture.stdout
result.stderr = result_capture.stderr
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel merged commit 4d9efd7 into main Mar 14, 2026
6 checks passed
@danielmeppiel danielmeppiel deleted the fix/windows-unicode-integration-tests branch March 14, 2026 11:53
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Windows integration/E2E tests against Unicode encoding/decoding issues by switching test log markers to ASCII and forcing UTF-8 decoding for subprocess output that captures APM CLI output.

Changes:

  • Replace Unicode markers in test print() statements (e.g., , ) with ASCII equivalents (e.g., [OK]).
  • Add encoding='utf-8' and errors='replace' to subprocess.run(...) / Popen(...) calls that capture text output.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tests/integration/test_guardrailing_hero_e2e.py Uses UTF-8 decoding for subprocess output and replaces Unicode log markers with ASCII.
tests/integration/test_mcp_registry_e2e.py Uses UTF-8 decoding for subprocess output and replaces Unicode log markers with ASCII.
tests/integration/test_ado_e2e.py Uses UTF-8 decoding for captured subprocess output (preemptive Windows hardening).
Comments suppressed due to low confidence (1)

tests/integration/test_mcp_registry_e2e.py:154

  • These logs still print result.stdout directly. apm mcp search/show uses Rich Table(...) output (box-drawing characters) and other commands emit Unicode symbols (e.g., "✓" in install output). On Windows with a cp1252 stdout, printing that captured text can still raise UnicodeEncodeError even after replacing the leading test marker. Consider sanitizing before printing (encode to the current stdout encoding with errors='replace'/backslashreplace), or avoid printing captured stdout in the test output.
        print(f"[OK] MCP search found GitHub servers:\n{result.stdout[:500]}...")
        

assert "installation" in output or "install" in output or "command" in output, "Should show installation info"

print(f" MCP show displays server details:\n{result.stdout[:500]}...")
print(f"[OK] MCP show displays server details:\n{result.stdout[:500]}...")
Comment on lines 205 to +209
text=True,
cwd=project_dir,
env=env
env=env,
encoding='utf-8',
errors='replace'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants