Problem
The playground editor at https://github.github.com/gh-aw/editor/ broke silently due to esm.sh CDN resolving duplicate @codemirror/state versions (see #25381). It was discovered manually — no test or monitoring caught it.
Root cause of the gap: Playwright tests exist in docs/tests/ (homepage links, copy button, mermaid rendering, mobile responsive) but:
- None cover the editor playground (
/editor/)
- None run in CI —
docs.yml builds and deploys without executing npm run test
Proposal
1. New Playwright spec: docs/tests/editor-playground.spec.ts
A minimal smoke test that:
- Navigates to
/gh-aw/editor/
- Waits for the WASM compiler to load (status badge text becomes "Ready")
- Verifies the CodeMirror editor mounted (
.cm-editor element exists with content)
- Asserts no console errors matching
Unrecognized extension or multiple instances
- Optionally: triggers a compile and verifies the output panel renders YAML
2. Wire Playwright into docs.yml
Add a test step (or job) after the build:
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
working-directory: ./docs
- name: Run Playwright tests
run: npx playwright test
working-directory: ./docs
The existing docs/playwright.config.ts already configures a webServer to start Astro dev on port 4321, so tests are self-contained.
3. Upload test artifacts on failure
- name: Upload Playwright report
if: failure()
uses: actions/upload-artifact@v4
with:
name: playwright-report
path: docs/playwright-report/
Why this matters
- The playground is a public-facing tool — silent breakage erodes trust
- CDN dependency drift (esm.sh) can break the editor at any time without a code change
- A 30-second Playwright test in CI would have caught this immediately
Related
Problem
The playground editor at https://github.github.com/gh-aw/editor/ broke silently due to esm.sh CDN resolving duplicate
@codemirror/stateversions (see #25381). It was discovered manually — no test or monitoring caught it.Root cause of the gap: Playwright tests exist in
docs/tests/(homepage links, copy button, mermaid rendering, mobile responsive) but:/editor/)docs.ymlbuilds and deploys without executingnpm run testProposal
1. New Playwright spec:
docs/tests/editor-playground.spec.tsA minimal smoke test that:
/gh-aw/editor/.cm-editorelement exists with content)Unrecognized extensionormultiple instances2. Wire Playwright into
docs.ymlAdd a test step (or job) after the build:
The existing
docs/playwright.config.tsalready configures awebServerto start Astro dev on port 4321, so tests are self-contained.3. Upload test artifacts on failure
Why this matters
Related
@codemirror/stateversions causing the current breakage