Skip to content

Add Playwright smoke test for editor playground in docs CI #25383

@Mossaka

Description

@Mossaka

Problem

The playground editor at https://github.github.com/gh-aw/editor/ broke silently due to esm.sh CDN resolving duplicate @codemirror/state versions (see #25381). It was discovered manually — no test or monitoring caught it.

Root cause of the gap: Playwright tests exist in docs/tests/ (homepage links, copy button, mermaid rendering, mobile responsive) but:

  1. None cover the editor playground (/editor/)
  2. None run in CIdocs.yml builds and deploys without executing npm run test

Proposal

1. New Playwright spec: docs/tests/editor-playground.spec.ts

A minimal smoke test that:

  • Navigates to /gh-aw/editor/
  • Waits for the WASM compiler to load (status badge text becomes "Ready")
  • Verifies the CodeMirror editor mounted (.cm-editor element exists with content)
  • Asserts no console errors matching Unrecognized extension or multiple instances
  • Optionally: triggers a compile and verifies the output panel renders YAML

2. Wire Playwright into docs.yml

Add a test step (or job) after the build:

- name: Install Playwright browsers
  run: npx playwright install --with-deps chromium
  working-directory: ./docs

- name: Run Playwright tests
  run: npx playwright test
  working-directory: ./docs

The existing docs/playwright.config.ts already configures a webServer to start Astro dev on port 4321, so tests are self-contained.

3. Upload test artifacts on failure

- name: Upload Playwright report
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: playwright-report
    path: docs/playwright-report/

Why this matters

  • The playground is a public-facing tool — silent breakage erodes trust
  • CDN dependency drift (esm.sh) can break the editor at any time without a code change
  • A 30-second Playwright test in CI would have caught this immediately

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions