Migrate block deletion E2E tests to Playwright#48012
Merged
Merged
Conversation
|
Size Change: 0 B Total Size: 1.33 MB ℹ️ View Unchanged
|
WunderBart
force-pushed
the
migrate/playwright-block-deletion
branch
2 times, most recently
from
f5fcc04 to
5e70010
Compare
WunderBart
added
[Type] Automated Testing
WunderBart
requested review from
ajitbohra,
kevin940726,
nerrad and
ntwb
as code owners
kevin940726
reviewed
Feb 14, 2023
Member
kevin940726
left a comment
kevin940726
left a comment
There was a problem hiding this comment.
Thank you! Really well written tests! Just left some nitpicks but overall it's good!
Note that changing the test titles means that the flaky tests reporter will no longer recognize them as the same test. If there are existing flaky test reports, we might want to close them after merging this.
|
Flaky tests detected in 9b27419. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/4281813641
|
WunderBart
force-pushed
the
migrate/playwright-block-deletion
branch
2 times, most recently
from
42495d5 to
a0d650a
Compare
Member
Author
GTK, thanks @kevin940726! To be sure, do the flaky reports depend on the title only or also on the filename (since we migrate from |
Member
Author
|
@kevin940726 performance specs are consistently failing with gutenberg/test/e2e/specs/editor/various/block-deletion.spec.js Lines 295 to 310 in aaf1276 |
Member
Nope, they only depend on test title.
It's the same WordPress instance but a different browser instance. I don't think they will impact each other, unless |
Member
Author
Looks like it's not persistent. It also passes locally, unfortunately. 😞 I'll rerun and keep debugging. |
Member
|
No wait, it's not even the same WordPress instance on CI 😅. |
Member
Author
I'm seeing the same perf specs failing on other branches as well (e.g. here and here), so it's not related to the current changes. It started happening just now, so I checked what changed recently and found 2 potential candidates: #47889 and #47777. The latter introduced a check that I'd normally mark as potentially flaky which might be causing the current issue. I'm trying out a fix on this PR here: 03d5b02. /cc @youknowriad edit: Testing in 2 other PRs as well: |
Contributor
|
@WunderBart I have a similar suspicion as well. I opened this to try to debug it #48094 |
Contributor
|
For some reason, my fix didn't work, we can try to go with yours if it works consistently. |
Contributor
|
It looks like your proposed fix doesn't fix it either, so it must be something else. |
WunderBart
force-pushed
the
migrate/playwright-block-deletion
branch
from
03d5b02 to
3f02486
Compare
kevin940726
approved these changes
Feb 22, 2023
Member
kevin940726
left a comment
kevin940726
left a comment
There was a problem hiding this comment.
Thanks again! A couple of suggestions but none of blocking 🙂 .
Comment on lines
+231
to
+233
| await expect( | ||
| editor.canvas.locator( '.is-multi-selected' ) | ||
| ).toHaveCount( 2 ); |
Member
There was a problem hiding this comment.
Nit: I did a similar thing in #48035 using the getMultiSelectedBlocks API. Maybe we can do something like that here?
await expect.poll( () =>
window.wp.data.select('core/block-editor').getMultiSelectedBlocks()
).toHaveCount( 2 );
Member
Author
There was a problem hiding this comment.
I guess we're not testing the multi-block selection here, so it's OK to validate programmatically! 👍
Member
There was a problem hiding this comment.
Even if we do, I don't think .is-multi-selected should be a public API for end users, so it should never be a testing target IMO.
Member
Author
There was a problem hiding this comment.
Could you elaborate on why you think it should not be a public API?
Member
There was a problem hiding this comment.
It's a class that's never exposed to the users, nor to the plugin developers. On the other hand, getMultiSelectedBlocks is a documented API. While it's still not a user-facing public API, but it's better than nothing I guess. A better way might be to check for accessible attributes, like aria-selected or aria-current, but that's probably not going to be available. The next option is visual testing, but not more difficult, and should only be tested once.
Co-authored-by: Kai Hao <kevin830726@gmail.com>
Not needed since Playwright v1.28
WunderBart
force-pushed
the
migrate/playwright-block-deletion
branch
from
ad44813 to
9b27419
Compare
Contributor
|
I just cherry-picked this PR to the wp/6.2 branch to get it included in the next release: caf09f1 |
ntsekouras
pushed a commit
that referenced
this pull request
Mar 7, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Migrate block deletion E2E tests to Playwright.
Why?
How?
Testing Instructions
The output of the following should be all ✅: