Skip to content

Run frontend tests with /ether plugin set on Firefox too #7621

@JohnMcLear

Description

@JohnMcLear

Follow-up to #7608 / PR #7609.

PR #7609 added playwright-chrome-with-plugins (gating, green). It originally also added playwright-firefox-with-plugins, but Firefox-with-plugins surfaced 23 hard failures with very different patterns from Chrome (full list below) and the job was dropped from the merging PR to keep CI clean. This issue tracks restoring it.

Change type: N/A (tracking issue)

Why Firefox is harder

Firefox + Etherpad's existing test suite is already brittle even without plugins — see commit db7a3575c ("fix: stabilize frontend tests and drop webkit from CI"). Adding 11 plugins doubles the boot time and adds extra hooks racing against fragile selectors; the 5-retry budget the with-plugins suite uses isn't enough.

The failure profile is browser-specific, not plugin-specific: most of these specs pass on Chrome+plugins. So adding WITH_PLUGINS && firefox skips would mask Firefox-only flake while preserving Chrome coverage — which is the wrong trade. Better to fix the underlying Firefox-with-plugins flakes.

Failures observed (PR #7609 commit fd66bdd5c, Firefox)

[firefox] alphabet.spec.ts:12   when you enter any char it appears right
[firefox] bold.spec.ts:12       makes text bold on click
[firefox] chat.spec.ts:63       chat sticks to right side via settings
[firefox] chat.spec.ts:123      chat icon click reveals chatbox after disable→enable
[firefox] delete.spec.ts:10     delete keystroke
[firefox] indentation.spec.ts:33    keeps indent on enter for new line
[firefox] indentation.spec.ts:141   shows '*' when multiple indented lines outdented
[firefox] ordered_list.spec.ts:31   keeps numbered list on enter
[firefox] page_up_down.spec.ts:12   PageDown moves caret forward
[firefox] select_focus_restore.spec.ts:8  toolbar select returns focus to pad (#7589)
[firefox] timeslider_line_numbers.spec.ts:10   line numbers aligned with rendered lines
[firefox] unaccepted_commit_warning.spec.ts:5  hasUnacceptedCommit clears on ack
[firefox] unordered_list.spec.ts:52   keeps UL on enter
[firefox] urls_become_clickable.spec.ts:10 (3 sub-tests)  URL → link
[firefox] urls_become_clickable.spec.ts:47 (~10 sub-tests) punctuation after URL

Total: 23 failed, 3 flaky, 41 skipped, 82 passed in 34.7m.

Path forward

  1. Decide whether Firefox is a tier-1 target for plugin coverage. If not, run this job nightly via schedule: instead of on every PR.
  2. If it stays on PRs: bisect each failure against the 11 plugins to identify hot spots (likely candidates: ep_align, ep_headings2, ep_subscript_and_superscript for indent/list issues; the URL-clickable ones probably hit a Firefox-specific paste handler).
  3. Patch failing specs the same way language.spec.ts was patched in ci: run frontend tests with /ether plugin set (closes #7608) #7609 (target by ID instead of index) where applicable.

How to reproduce

pnpm install --frozen-lockfile
pnpm add -w ep_align ep_author_hover ep_cursortrace ep_font_size ep_headings2 ep_markdown ep_readonly_guest ep_set_title_on_pad ep_spellcheck ep_subscript_and_superscript ep_table_of_contents
cp src/tests/settings.json settings.json
pnpm run prod &
sleep 15
cd src
pnpm exec playwright install firefox
WITH_PLUGINS=1 pnpm run test-ui --project=firefox

When fixed: re-add the playwright-firefox-with-plugins job to .github/workflows/frontend-tests.yml (mirroring playwright-chrome-with-plugins), and reference this issue + #7611 in the PR.

🤖 Filed via PR #7609.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions