Skip to content

ci(frontend-tests): exclude ep_cursortrace + un-flake 30 of 31 #7611 skips#7630

Merged
JohnMcLear merged 23 commits intoether:developfrom
JohnMcLear:fix/unflake-with-plugins-skips
Apr 29, 2026
Merged

ci(frontend-tests): exclude ep_cursortrace + un-flake 30 of 31 #7611 skips#7630
JohnMcLear merged 23 commits intoether:developfrom
JohnMcLear:fix/unflake-with-plugins-skips

Conversation

@JohnMcLear
Copy link
Copy Markdown
Member

@JohnMcLear JohnMcLear commented Apr 29, 2026

Summary

Two changes that together un-flake the Playwright * with plugins CI matrix:

  1. Drop ep_cursortrace from frontend-tests.yml's WITH_PLUGINS plugin set. Bisected as the sole cause of the Firefox-with-plugins flakiness this PR was chasing. Its aceEditEvent hook fires per keyboard event and unconditionally sends a socket cursorPosition message — under the test harness's writeToPad bursts that stream saturates the editor's input pipeline in Firefox, dropping keystrokes and producing the entire class of Frontend specs that fail with /ether plugin set loaded #7611 symptoms. The plugin itself needs a debounce around the socket send before it can come back into the test set; that fix lives outside this PR.

  2. Remove 30 of 31 test.skip(WITH_PLUGINS, '#7611') markers across the src/tests/frontend-new/specs/ tree. With ep_cursortrace out of the plugin set, those tests pass under WITH_PLUGINS=1. The remaining 1 skip is timeslider_follow #4389, which builds ~120 sequential Enter keypresses — needs a different setup mechanism (REST-driven import or clipboard paste) regardless of which plugins are loaded.

Change type: patch (CI/test infrastructure only; no production behaviour change).

Bisection (4 iterations on this branch)

Iter Plugin sub-set Firefox+plugins result
1 A: align, author_hover, cursortrace, font_size, headings2 ❌ fails
1 B: markdown, readonly_guest, set_title_on_pad, spellcheck, subscript_and_superscript, table_of_contents ✅ passes
2 A1: align, author_hover ✅ passes
2 A2: cursortrace, font_size, headings2 ❌ fails
3 A2a: cursortrace ❌ fails (1 plugin alone tips it)
3 A2b: font_size, headings2 ✅ passes
4 All 10 minus cursortrace ✅ passes (×2 independent Firefox runs)

Helper changes (carry-overs from before the bisection landed)

These all stay because they're either independent improvements or precondition fixes once the cursortrace load is gone:

  • helper/padHelper.ts: added waitForEditorReady used by goToNewPad / goToPad. Blocks until #innerdocbody[contenteditable="true"]. Without this, specs that started interacting immediately after #editorcontainer.initialized would race the ace static→editable flip.
  • Per-spec swaps from page.keyboard.type to keyboard.insertText / writeToPad (continuation of test(playwright): use insertText so Firefox stops dropping keystrokes #7625's logic). Reliable in Firefox under any non-cursortrace plugin load.
  • Per-spec force:true on toolbar button clicks after selectAllText (matches the existing clearAuthorship pattern for the #toolbar-overlay interception).
  • enter.spec.ts: replaced an unverified Enter-press loop with a per-iteration expect(...).toHaveCount(...) so the loop doesn't advance until the previous press has landed.
  • ordered_list.spec.ts: switched the OL toolbar selector from .buttonicon-insertorderedlist.first() to button[data-l10n-id='pad.toolbar.ol.title'] (per Qodo review — plugin-resilient, no .first() index assumption).

Test plan

  • Bisection confirmed via 4 CI iterations.
  • Final 10-plugin set passes both Chrome+plugins and Firefox+plugins (also 2× independent Firefox confirmation runs).
  • No regressions on Chromium / no-plugins.
  • Locally verified all batches on Firefox + WITH_PLUGINS=1 (against the older test setup that included cursortrace; the patterns the helper/spec changes establish are still correct).

Followups (not blockers for this PR)

  • Add a mousemove/keypress debounce to ep_cursortrace's aceEditEvent so it can return to the test plugin set.
  • timeslider_follow.spec.ts #4389 — only remaining #7611 skip; needs a non-keyboard setup path.
  • Several core specs (Linux with Plugins (24) SessionStore.ts shutdown cancels timeouts) hit a separate timing flake unrelated to this work; saw it fire ~3× across this session's CI runs.

🤖 Generated with Claude Code

@JohnMcLear JohnMcLear changed the title test(playwright): un-flake WITH_PLUGINS skipped specs (incremental) test(playwright): un-flake all WITH_PLUGINS skipped specs (closes #7611) Apr 29, 2026
@JohnMcLear JohnMcLear marked this pull request as ready for review April 29, 2026 08:43
@qodo-free-for-open-source-projects
Copy link
Copy Markdown

Review Summary by Qodo

Un-flake 31 WITH_PLUGINS-skipped tests by fixing editor readiness and input reliability

🧪 Tests

Grey Divider

Walkthroughs

Description
• Add waitForEditorReady helper to ensure editor is editable before tests run
• Remove 31 test.skip(WITH_PLUGINS) markers across 21 spec files
• Replace per-character keyboard.type with keyboard.insertText for reliable input
• Add {force: true} to toolbar button clicks after text selection to bypass overlay
• Refactor loops with per-iteration verification to prevent dropped keystroke events
Diagram
flowchart LR
  A["Editor initialization"] -->|"waitForEditorReady"| B["Verify contenteditable=true"]
  C["Per-char keyboard.type"] -->|"Replace with insertText"| D["Single input event"]
  E["Toolbar clicks after selection"] -->|"Add force:true"| F["Bypass toolbar-overlay"]
  G["Tight keystroke loops"] -->|"Add per-iteration verify"| H["Prevent dropped events"]
  B --> I["31 tests un-skipped"]
  D --> I
  F --> I
  H --> I
Loading

Grey Divider

File Changes

1. src/tests/frontend-new/helper/padHelper.ts ✨ Enhancement +20/-4

Add waitForEditorReady helper for editor editability

src/tests/frontend-new/helper/padHelper.ts


2. src/tests/frontend-new/specs/bold.spec.ts 🐞 Bug fix +20/-15

Un-skip tests, add force:true, use writeToPad

src/tests/frontend-new/specs/bold.spec.ts


3. src/tests/frontend-new/specs/alphabet.spec.ts 🐞 Bug fix +5/-5

Un-skip test, replace keyboard.type with writeToPad

src/tests/frontend-new/specs/alphabet.spec.ts


View more (18)
4. src/tests/frontend-new/specs/delete.spec.ts 🐞 Bug fix +5/-3

Un-skip test, use writeToPad for input

src/tests/frontend-new/specs/delete.spec.ts


5. src/tests/frontend-new/specs/enter.spec.ts 🐞 Bug fix +10/-9

Un-skip test, add per-iteration line count verification

src/tests/frontend-new/specs/enter.spec.ts


6. src/tests/frontend-new/specs/indentation.spec.ts 🐞 Bug fix +22/-29

Un-skip tests, add force:true, use insertText/writeToPad

src/tests/frontend-new/specs/indentation.spec.ts


7. src/tests/frontend-new/specs/ordered_list.spec.ts 🐞 Bug fix +8/-14

Un-skip tests, add force:true, consolidate multi-line input

src/tests/frontend-new/specs/ordered_list.spec.ts


8. src/tests/frontend-new/specs/unordered_list.spec.ts 🐞 Bug fix +9/-7

Un-skip test, add force:true, use writeToPad

src/tests/frontend-new/specs/unordered_list.spec.ts


9. src/tests/frontend-new/specs/list_wrap_indent.spec.ts 🐞 Bug fix +4/-2

Un-skip test, add force:true to button click

src/tests/frontend-new/specs/list_wrap_indent.spec.ts


10. src/tests/frontend-new/specs/undo_redo_scroll.spec.ts 🐞 Bug fix +15/-16

Un-skip tests, replace loop with multi-line writeToPad

src/tests/frontend-new/specs/undo_redo_scroll.spec.ts


11. src/tests/frontend-new/specs/collab_client.spec.ts 🐞 Bug fix +4/-2

Un-skip test, replace keyboard.type with insertText

src/tests/frontend-new/specs/collab_client.spec.ts


12. src/tests/frontend-new/specs/undo_clear_authorship.spec.ts 🐞 Bug fix +4/-3

Un-skip test, use insertText for reliable input

src/tests/frontend-new/specs/undo_clear_authorship.spec.ts


13. src/tests/frontend-new/specs/page_up_down.spec.ts 🐞 Bug fix +0/-3

Un-skip three tests, skip removal only

src/tests/frontend-new/specs/page_up_down.spec.ts


14. src/tests/frontend-new/specs/chat.spec.ts 🐞 Bug fix +0/-2

Un-skip two tests, skip removal only

src/tests/frontend-new/specs/chat.spec.ts


15. src/tests/frontend-new/specs/clear_authorship_color.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/clear_authorship_color.spec.ts


16. src/tests/frontend-new/specs/bold_paste.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/bold_paste.spec.ts


17. src/tests/frontend-new/specs/select_focus_restore.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/select_focus_restore.spec.ts


18. src/tests/frontend-new/specs/timeslider_follow.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/timeslider_follow.spec.ts


19. src/tests/frontend-new/specs/timeslider_line_numbers.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/timeslider_line_numbers.spec.ts


20. src/tests/frontend-new/specs/unaccepted_commit_warning.spec.ts 🐞 Bug fix +0/-1

Un-skip test, skip removal only

src/tests/frontend-new/specs/unaccepted_commit_warning.spec.ts


21. src/tests/frontend-new/specs/urls_become_clickable.spec.ts 🐞 Bug fix +0/-1

Un-skip file-level test, skip removal only

src/tests/frontend-new/specs/urls_become_clickable.spec.ts


Grey Divider

Qodo Logo

@qodo-free-for-open-source-projects
Copy link
Copy Markdown

qodo-free-for-open-source-projects Bot commented Apr 29, 2026

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider


Action required

1. $insertorderedlistButton.first() index use📎 Requirement gap ☼ Reliability
Description
The updated ordered list spec still relies on .first() to choose a toolbar button match, which is
DOM-order dependent and can change under plugins. This violates the guideline to avoid
plugin-sensitive selector/index assumptions.
Code

src/tests/frontend-new/specs/ordered_list.spec.ts[R16-21]

+      // force:true bypasses #toolbar-overlay (intercepts pointer
+      // events after a text selection); same pattern as
+      // clearAuthorship.
const $insertorderedlistButton = page.locator('.buttonicon-insertorderedlist')
await padBody.locator('div').first().selectText()
-      await $insertorderedlistButton.first().click();
+      await $insertorderedlistButton.first().click({force: true});
Evidence
PR Compliance ID 2 requires selectors that remain stable when plugins modify the UI; the modified
code uses $insertorderedlistButton.first() which is explicitly index-based and can select the
wrong element if additional matching nodes are introduced.

Avoid plugin-sensitive selector/index assumptions in frontend specs
src/tests/frontend-new/specs/ordered_list.spec.ts[16-21]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The spec clicks the ordered-list toolbar button via `$insertorderedlistButton.first()`, which is order-dependent and may break when plugins alter the toolbar DOM.
## Issue Context
Prefer a uniquely identifying selector (for example, `button[data-l10n-id='pad.toolbar.ol.title']` or another stable attribute that does not depend on element order).
## Fix Focus Areas
- src/tests/frontend-new/specs/ordered_list.spec.ts[16-21]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Advisory comments

2. Misleading locator variable name 🐞 Bug ⚙ Maintainability
Description
In unordered_list.spec.ts, the variable $insertorderedlistButton actually locates the unordered
list toolbar button, which makes the test harder to read and increases the chance of future edits
using the wrong action.
Code

src/tests/frontend-new/specs/unordered_list.spec.ts[R59-60]

const $insertorderedlistButton = page.locator('.buttonicon-insertunorderedlist')
-      await $insertorderedlistButton.click();
+      await $insertorderedlistButton.click({force: true});
Evidence
The edited block introduces/uses $insertorderedlistButton for .buttonicon-insertunorderedlist,
while the same file already uses the clearer $insertunorderedlistButton name elsewhere, creating
inconsistent naming for the same control.

src/tests/frontend-new/specs/unordered_list.spec.ts[57-61]
src/tests/frontend-new/specs/unordered_list.spec.ts[15-18]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`$insertorderedlistButton` is used to refer to `.buttonicon-insertunorderedlist`, which is misleading in an unordered list spec.
### Issue Context
This is a readability/maintainability issue that can cause confusion during future test edits.
### Fix Focus Areas
- src/tests/frontend-new/specs/unordered_list.spec.ts[57-61]
- src/tests/frontend-new/specs/unordered_list.spec.ts[15-18]
### Suggested change
Rename `$insertorderedlistButton` to `$insertunorderedlistButton` in the affected test block (and update its uses) for consistency with the rest of the file.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread src/tests/frontend-new/specs/ordered_list.spec.ts Outdated
@JohnMcLear
Copy link
Copy Markdown
Member Author

Addressed in 7157af1: replaced page.locator('.buttonicon-insertorderedlist').first() with page.locator("button[data-l10n-id='pad.toolbar.ol.title']") — the localizationId declared in src/node/utils/toolbar.ts for the orderedlist button. Unique by construction, no .first() needed, plugin-resilient.

Left the class-based locator in #5160, #5718, and the indent/outdent sub-describes since they don't currently strict-mode-match more than one element. Happy to swap those too for consistency if you'd prefer.

JohnMcLear added a commit to JohnMcLear/etherpad that referenced this pull request Apr 29, 2026
…rlay regressions

Three fixes for the failures that surfaced once ether#7630 ran in CI on
Firefox + WITH_PLUGINS at the full matrix:

1. **writeToPad** now value-waits per Enter and retries up to 3
   times if the editor doesn't acknowledge a new line. Long
   multi-line writes (e.g. timeslider_follow's ether#4389 setup with
   ~120 newlines) were dropping Enters faster than the previous
   single-press loop tolerated. The retry surfaces the canonical
   "expected N, got M" timeout if all 3 attempts fail.

2. **unordered_list.spec.ts**: every `.buttonicon-*` toolbar click
   now uses {force: true}. Two of the un-skipped tests intermittently
   missed the click under load because #toolbar-overlay intercepts
   pointer events after a text selection (same pattern as bold,
   ep_align, et al.). Body clicks (clicks inside the iframe pad
   body) are unaffected and stay as plain `.click()`.

3. **timeslider_follow.spec.ts** "regression test for ether#4389":
   re-skipped under WITH_PLUGINS with a specific note. The 120-Enter
   setup races plugin load even with the new writeToPad retry —
   re-press attempts overshoot the exact line count when a "dropped"
   Enter eventually lands. Needs a fundamentally different setup
   approach (REST API import, clipboard paste, etc.) to un-skip
   reliably; out of scope here.

Net: 30 of the original 31 ether#7611 skips remain removed (was 31/31
before; the one re-skip is a documented known-aggressive case).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JohnMcLear JohnMcLear changed the title test(playwright): un-flake all WITH_PLUGINS skipped specs (closes #7611) test(playwright): un-flake 23 of 31 WITH_PLUGINS skipped specs (refs #7611) Apr 29, 2026
JohnMcLear and others added 18 commits April 29, 2026 11:19
#editorcontainer.initialized fires after padeditor.init resolves but
before ace flips the inner body from `class="static"` /
contentEditable=false to editable. Under WITH_PLUGINS load in Firefox
that flip can lag long enough that an immediate click + keyboard.type
runs against a still-static body and is silently dropped — the body
keeps showing the default welcome text and never sees our input.

Most of the specs that currently carry `test.skip(WITH_PLUGINS)`
markers (ether#7611) are racing exactly this flip. Block in goToNewPad /
goToPad until the inner #innerdocbody is `contenteditable="true"`,
so every spec starts from a known-ready editor without each having
to add its own ad-hoc waits.

Value-driven: exits as soon as ace flips the attribute, no fixed
delay. Refactored into a private waitForEditorReady() helper so
goToNewPad and goToPad share a single source of truth.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two skipped tests fail because clicking the bold toolbar button
right after selectAllText is intercepted by the #toolbar-overlay div
(same root cause that needed force:true in clearAuthorship and
ep_align). Add force:true to the click and drop the
test.skip(WITH_PLUGINS) markers.

The keypress variant doesn't click a toolbar button — it relies on
the editor being editable when keyboard.press fires. The previous
commit (waitForEditorReady in goToNewPad) covers that.

Proof-of-concept un-skip; if CI confirms both pass, will expand the
same pattern to the rest of the ether#7611 skip set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous attempt at un-skipping these tests added force:true on
the toolbar click but left the legacy selectAllText + keyboard.type
sequence in place. Firefox under WITH_PLUGINS load racily drops
keystrokes from per-key events, leaving an empty selection that the
bold-on-click and Ctrl+B branches both no-op'd against — the asserts
then timed out 5 retries deep with no <b> element.

Replace the selectAllText + keyboard.type prelude with the standard
clearPadContent + writeToPad pair. writeToPad uses insertText (one
input event for the whole string) which is the same fix that
unblocked ep_align in ether#7625.

Verified locally on Firefox + WITH_PLUGINS=1: 2/2 pass in 15s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These four specs were marked test.skip(WITH_PLUGINS) for "flaky in
with-plugins suite" but only use writeToPad / clearPadContent /
goToNewPad — no direct keyboard.type, no toolbar button clicks. The
flake was the editor not being ready when the test's first
interaction fired (now covered by waitForEditorReady in
goToNewPad/goToPad earlier in this branch) plus writeToPad's switch
to insertText (ether#7625).

  - urls_become_clickable.spec.ts (file-level skip)
  - unaccepted_commit_warning.spec.ts
  - undo_clear_authorship.spec.ts
  - timeslider_follow.spec.ts

Just removing the skip lines is enough; no other changes needed.

Verified locally on Firefox + WITH_PLUGINS=1: all 40 tests across
the four specs pass in 3m1s. urls_become_clickable contributes the
bulk (37 tests via parameterised describes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…der WITH_PLUGINS

Both specs use writeToPad + keyboard.press for Page Up/Down, End,
arrow keys, and the like — no per-character keyboard.type, no
toolbar button clicks. The flake was the editor not being ready
when the spec's first interaction fired (now covered by
waitForEditorReady earlier in this branch) plus writeToPad's switch
to insertText (ether#7625) for the multi-line setup.

  - page_up_down.spec.ts (3 skips)
  - timeslider_line_numbers.spec.ts (1 skip)

Verified locally on Firefox + WITH_PLUGINS=1: 5/5 tests pass.

enter.spec.ts deliberately left skipped — its Enter-in-a-loop test
(line 33) drops keypresses under load and needs a value-driven
per-iteration verify, separate change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…do_clear_authorship

Three more files cleared after the editor-ready helper landed:

  - chat.spec.ts (2 skips) — both clicks target settings-popup
    checkboxes, not toolbar buttons; the toolbar-overlay isn't
    in play, so just dropping the skips is enough.
  - clear_authorship_color.spec.ts (1) — uses the existing
    clearAuthorship helper, which already runs with force:true.
  - list_wrap_indent.spec.ts (1) — adds force:true to the
    .buttonicon-insertorderedlist click that fires after
    selectAllText (same pattern as bold.spec).

Reverts the un-skip on undo_clear_authorship.spec.ts: that one
spawns two browser contexts and races against User B's writeToPad
landing in the second pad. Hit a real flake locally where User B's
text never appeared. Needs a per-user "wait for text to commit"
before the assertion. Re-add the skip until that fix is in.

Verified locally on Firefox + WITH_PLUGINS=1: 16 passed across
the three un-skipped files (one undo_clear_authorship retry
flaked, hence the revert).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…WITH_PLUGINS

  - alphabet.spec.ts (1) — swapped page.keyboard.type for writeToPad
  - delete.spec.ts (1) — same swap
  - select_focus_restore.spec.ts (1) — left keyboard.type in place
    (the test specifically verifies that focus returns to the editor
    after a toolbar select change; replacing with writeToPad would
    re-focus the body via a click and mask the bug being asserted).
    Editor-ready wait alone is enough here.

Verified locally on Firefox + WITH_PLUGINS=1: 3/3 pass in 23s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…UGINS

  - bold_paste.spec.ts (1) — already used writeToPad; just dropping
    the skip is enough now that the editor-ready helper landed.
  - undo_redo_scroll.spec.ts (2) — replaced the
    `for (45 lines) { keyboard.type; keyboard.press('Enter') }` loop
    with a single writeToPad of `lines.join('\\n') + '\\n'`. writeToPad
    drives input via insertText (one input event per line) which
    Firefox under WITH_PLUGINS load handles without dropping events.
    The Ctrl+Z scroll-to-caret behaviour the test asserts is
    unchanged — each line still lands in its own changeset for the
    undo module to reverse.

Verified locally on Firefox + WITH_PLUGINS=1: bold_paste passes
clean; undo_redo_scroll passes via the existing per-spec
`retries: 2` config (the scroll timing race exists pre-change and
is what motivates the retries).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…er WITH_PLUGINS

  - Add force:true on the .buttonicon-insertunorderedlist click to
    bypass #toolbar-overlay (same pattern as clearAuthorship and
    bold.spec).
  - Replace the
      keyboard.type('line 1'); keyboard.press('Enter');
      keyboard.type('line 2'); keyboard.press('Enter');
    sequence with a single writeToPad('line 1\\nline 2\\n') —
    insertText per line + Enter between, which Firefox under
    WITH_PLUGINS load handles without dropping events. The trailing
    newline preserves the final Enter the original spec relied on.

Verified locally on Firefox + WITH_PLUGINS=1: passes in 8s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  - issue ether#4748 + ether#1125: add force:true on
    .buttonicon-insertorderedlist clicks (toolbar-overlay
    interception after selection); collapse the per-line
    keyboard.type + keyboard.press('Enter') sequences into single
    writeToPad calls with embedded newlines.
  - issue ether#5160 and ether#5718 already used force:true and writeToPad
    throughout; just dropping the skip is enough now that the
    editor-ready helper landed.

Verified locally on Firefox + WITH_PLUGINS=1: 11 passed (4 ordered_list
+ 5 unordered_list, plus 2 sub-describes). 1m24s total.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pattern as bold/ordered_list/unordered_list:
  - force:true on .buttonicon-indent / .buttonicon-bold /
    .buttonicon-outdent clicks (toolbar-overlay interception
    after a text selection).
  - Replace per-line keyboard.type + keyboard.press('Enter')
    sequences with single writeToPad calls using \\n separators.
  - Replace single-character keyboard.type calls (':', '(', '[',
    '{{') with keyboard.insertText for consistency.

The keypress and indent/outdent button tests were already passing
without WITH_PLUGINS skips — only the four tests that race the
toolbar click + typing sequence were skipped. With force:true and
writeToPad they're stable.

Verified locally on Firefox + WITH_PLUGINS=1: 12 tests pass across
indentation, ordered_list, unordered_list, list_wrap_indent
(matched by the indent grep). 1m11s total.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…WITH_PLUGINS

The test fired 15 keypress('Enter') calls in a tight loop with no
per-iteration verify. Under Firefox + WITH_PLUGINS load the
editor's input pipeline can't always keep up while plugin hooks
are warming, so a few presses get dropped and the final
`expect(div.count).toBe(numberOfLines + originalLength)` fails
with too few lines.

Add a value-driven `expect(div).toHaveCount(originalLength + i + 1)`
after each press. The loop only advances once the editor has
acknowledged the previous Enter, so dropped events become slow
events instead of lost ones.

Verified locally on Firefox + WITH_PLUGINS=1: passes in 11s
(would have been 1.5m timeout previously).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two-user test was racing on User B's keyboard.type('Hello from
User B') and 'Still connected!' — Firefox + WITH_PLUGINS load drops
keystrokes from per-key events, leaving the second pad with
truncated text that the body1 round-trip assertion never matches.

Replace both keyboard.type calls with keyboard.insertText (single
input event). Cannot use writeToPad here because the test relies on
the caret position established by the preceding End + Enter — a
writeToPad would re-click the body and reset focus.

Verified locally on Firefox + WITH_PLUGINS=1: both tests pass clean
in 30s (previously failed all retries at 1m+ each). The
test.describe.configure({retries: 2}) is kept as belt-and-braces
for the multi-context server propagation race that this test
exercises legitimately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…st' under WITH_PLUGINS

The test's replaceLineText helper used keyboard.type(newText) to
insert the replacement string after a Backspace clear. Firefox under
WITH_PLUGINS load drops keystrokes from per-key events, leaving the
line with truncated text that the cross-context assertions
(body1.toHaveText(user2Text), body2.toHaveText(user1Text)) never
match.

Switch the type to keyboard.insertText (single input event) — same
fix that unblocked ep_align in ether#7625 and the other typing-races in
this branch. The selectText + Backspace + insertText pattern still
exercises the legitimate collab race the test asserts (concurrent
edits over the COLLABROOM).

Verified locally on Firefox + WITH_PLUGINS=1: passes in 15s.

This was the last of the 31 test.skip(WITH_PLUGINS, 'ether#7611') markers
in src/tests/frontend-new/specs/. The branch goal of zero ether#7611
skips is met.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Qodo flagged the .first() call in ether#4748's setup as DOM-order
dependent: a future plugin that adds another element carrying the
.buttonicon-insertorderedlist class would silently change which
button the test clicks. Switch to
button[data-l10n-id='pad.toolbar.ol.title'] (the localizationId
declared in src/node/utils/toolbar.ts), which is unique to the core
ordered-list toolbar entry. Drop the now-unnecessary .first().

The class-based locator remains in ether#5160, ether#5718, and the indent/
outdent sub-describes; those don't strict-mode-match more than one
element today, but a follow-up could swap them too for consistency
if reviewers want.

Verified locally on Firefox + WITH_PLUGINS=1: passes in 7s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rlay regressions

Three fixes for the failures that surfaced once ether#7630 ran in CI on
Firefox + WITH_PLUGINS at the full matrix:

1. **writeToPad** now value-waits per Enter and retries up to 3
   times if the editor doesn't acknowledge a new line. Long
   multi-line writes (e.g. timeslider_follow's ether#4389 setup with
   ~120 newlines) were dropping Enters faster than the previous
   single-press loop tolerated. The retry surfaces the canonical
   "expected N, got M" timeout if all 3 attempts fail.

2. **unordered_list.spec.ts**: every `.buttonicon-*` toolbar click
   now uses {force: true}. Two of the un-skipped tests intermittently
   missed the click under load because #toolbar-overlay intercepts
   pointer events after a text selection (same pattern as bold,
   ep_align, et al.). Body clicks (clicks inside the iframe pad
   body) are unaffected and stay as plain `.click()`.

3. **timeslider_follow.spec.ts** "regression test for ether#4389":
   re-skipped under WITH_PLUGINS with a specific note. The 120-Enter
   setup races plugin load even with the new writeToPad retry —
   re-press attempts overshoot the exact line count when a "dropped"
   Enter eventually lands. Needs a fundamentally different setup
   approach (REST API import, clipboard paste, etc.) to un-skip
   reliably; out of scope here.

Net: 30 of the original 31 ether#7611 skips remain removed (was 31/31
before; the one re-skip is a documented known-aggressive case).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e more failures

The per-Enter value-wait + retry I added in fc45d71 was meant to
catch dropped Enters in long multi-line writes, but in CI it made
things worse: when a "dropped" Enter eventually landed during the
retry's short poll window, the next iteration's exact line-count
expectation was off by one and the retry loop overshot, breaking
tests that previously passed (urls_become_clickable, language,
inner_height all hit toHaveCount mismatches that didn't exist
before).

Revert to the simpler insertText + bare keyboard.press('Enter')
loop. Tests with extreme line counts (timeslider_follow ether#4389,
~120 Enters) stay re-skipped from the prior commit; everything
else accepts the same intermittent flake the helper exhibited
before this fix attempt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Honest scope adjustment after CI surfaced load-dependent failures
that local single-run verification missed. The previous batches
worked at low concurrency but flake at the full Playwright matrix
under WITH_PLUGINS:

  - bold_paste.spec.ts — clipboard / paste race between specs
  - collab_client.spec.ts (bug ether#4978) — multi-context cross-pad
    propagation under load
  - enter.spec.ts (enter is always visible) — 15-Enter loop drops
    presses faster than the per-iteration value-wait can recover
  - timeslider_follow.spec.ts (content as it's added) — 66 sequential
    Enters across 6 writeToPad calls
  - undo_clear_authorship.spec.ts (describe-level) — multi-context;
    the cross-pad text-arrival assertion races
  - undo_redo_scroll.spec.ts (describe-level) — 45-line writeToPad
    setup; scroll-position assertion needs stable layout
  - unordered_list.spec.ts (Keeps unordered list on enter) — toolbar
    click + writeToPad with embedded newline races

All carry inline comments explaining the specific load issue and
referencing ether#7611 so a follow-up that introduces a REST-driven or
clipboard-paste setup mechanism can target them concretely.

Net: 23 of 31 ether#7611 skips removed (74%). The deferred 8 share two
underlying limitations that need infrastructure work:
  1. No reliable way to drive >10 sequential Enters under load
     without occasional drops
  2. No reliable cross-context propagation wait helper

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JohnMcLear JohnMcLear force-pushed the fix/unflake-with-plugins-skips branch from 686ba33 to 9381490 Compare April 29, 2026 10:19
JohnMcLear and others added 3 commits April 29, 2026 11:31
One CI run, both halves of the standard plugin set, both on Firefox
(which is the project that reliably trips the flake we're chasing).

  Playwright Firefox with plugins  → HALF A: ep_align, ep_author_hover,
                                      ep_cursortrace, ep_font_size,
                                      ep_headings2
  Playwright Chrome with plugins   → HALF B: ep_markdown, ep_readonly_guest,
                                      ep_set_title_on_pad, ep_spellcheck,
                                      ep_subscript_and_superscript,
                                      ep_table_of_contents
                                      (job runs --project=firefox here too)

Decision matrix on next CI:
  - Both fail        → load alone is the cause; deeper rework needed.
  - Only A fails     → culprit is in HALF A (5 candidates).
  - Only B fails     → culprit is in HALF B (6 candidates).
  - Both pass        → flake threshold sits between 5–6 plugins; the
                        culprit is whichever 2-plugin pair from the full
                        set tips the load above threshold; iterate.

Revert this commit before merging — it's purely a CI probe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rsortrace,font_size,headings2)

Iteration 1 isolated to HALF A. Splitting:
  Playwright Firefox with plugins → A1: ep_align, ep_author_hover
  Playwright Chrome with plugins  → A2: ep_cursortrace, ep_font_size,
                                         ep_headings2 (still --project=firefox)

Decision matrix:
  - Both fail        → load alone tips it; ≥2 of these 5 are needed.
  - Only A1 fails    → culprit is ep_align or ep_author_hover.
  - Only A2 fails    → culprit is ep_cursortrace, ep_font_size, or ep_headings2.
  - Both pass        → flake threshold is between 2 and 3 plugins from A,
                        revisit splitting (could be a specific pair).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ze, headings2)

Iteration 2 isolated to A2 (cursortrace+font_size+headings2).
Iter 3 singles out ep_cursortrace:

  Playwright Firefox with plugins → A2a: ep_cursortrace
  Playwright Chrome with plugins  → A2b: ep_font_size, ep_headings2
                                         (still --project=firefox)

Decision matrix:
  - Only A2a fails   → ep_cursortrace is the culprit (1 plugin alone tips it).
  - Only A2b fails   → culprit is ep_font_size or ep_headings2.
  - Both fail        → load tips at >=1 plugin from this set; investigate
                        each individually.
  - Both pass        → load tips at >=3 plugins; revisit splitting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JohnMcLear and others added 2 commits April 29, 2026 12:02
Iter 3 isolated to ep_cursortrace alone. Confirming by running the
inverse — every other plugin in the standard set, no ep_cursortrace —
on TWO Firefox runs in parallel:

  Playwright Firefox with plugins → align, author_hover, font_size,
                                     headings2, markdown,
                                     readonly_guest, set_title_on_pad,
                                     spellcheck,
                                     subscript_and_superscript,
                                     table_of_contents
  Playwright Chrome with plugins  → same 10 plugins (still
                                     --project=firefox per probe)

Both pass → ep_cursortrace is conclusively the culprit.
Either fails → load is the cause and the bisection mis-attributed
              (would need to investigate why iter 3 cursortrace-only
              failed: maybe a flaky one-off).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bisected via 4 CI iterations on this branch. ep_cursortrace's
`aceEditEvent` hook (static/js/main.js in the plugin) fires on every
keyboard event — handleClick, handleKeyEvent, idleWorkTimer — and
unconditionally sends a `cursorPosition` socket message via
`pad.collabClient.sendMessage` per call. Under the test harness's
writeToPad bursts (insertText + Enter loops) that stream of socket
messages saturates the editor's input pipeline in Firefox
specifically, causing intermittent keystroke drops and the entire
class of ether#7611 flakiness this PR was originally chasing.

Confirmation runs:
  - 11-plugin set including ep_cursortrace            → fails on Firefox
  - HALF B (5 plugins, no cursortrace)                → passes
  - HALF A (5 plugins, with cursortrace)              → fails
  - A1 (align, author_hover) — no cursortrace         → passes
  - A2 (cursortrace, font_size, headings2)            → fails
  - A2a (cursortrace alone, 1 plugin)                 → fails
  - A2b (font_size, headings2, no cursortrace)        → passes
  - 10-plugin set, all minus ep_cursortrace           → passes (×2 jobs)

Drop ep_cursortrace from the frontend-tests.yml plugin set and
restore all the un-skips that this PR pessimistically re-skipped
during the load-symptom whack-a-mole. The plugin itself needs a
debounce/throttle around its socket send before it can come back
into the test set; tracked separately in the ep_cursortrace repo.

Backend tests / docker / etc remain on the original 11-plugin set
since they don't trip the same input-pipeline race.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JohnMcLear JohnMcLear changed the title test(playwright): un-flake 23 of 31 WITH_PLUGINS skipped specs (refs #7611) ci(frontend-tests): exclude ep_cursortrace + un-flake 30 of 31 #7611 skips Apr 29, 2026
@JohnMcLear JohnMcLear merged commit bbd2968 into ether:develop Apr 29, 2026
18 checks passed
JohnMcLear added a commit that referenced this pull request Apr 30, 2026
Remove the FRONTEND_IGNORE entry that suppressed
ep_headings2/static/tests/frontend-new/specs/headings.spec.ts under
WITH_PLUGINS=1. The skip was added in #7628 while the keystroke-drop
flake (#7611) was still being chased; #7630 then identified the actual
root cause as ep_cursortrace's per-keystroke cursorPosition socket
spam saturating Firefox's input pipeline, removed ep_cursortrace
from the WITH_PLUGINS plugin set, and added waitForEditorReady() to
goToNewPad/goToPad. With both root causes addressed, this skip is
likely stale — the spec's own "Option select is changed when heading
is changed" test already uses insertText for the second-line typing,
so it should clear the same bar that #7630 cleared for ep_markdown
and ep_spellcheck (both now passing on develop).

Closes #7626 if CI confirms — the issue's three plugin
specs (markdown, spellcheck, headings2) and timeslider_identity_changeset
are all addressed once this lands. If headings2 is still flaky after
this, FRONTEND_IGNORE comes back with a narrower comment about what
specifically still races.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant