fix(test runner): do not write missing snapshot until the last retry#9246
Merged
Conversation
This prevents future retries from passing because of the actual snapshot being written. In theory, we can avoid running the retry since it should fail anyway. However, this brings problems, for example in the `describe.serial` mode where running a test also has some side effects and so it should not be skipped. Since running a test without a snapshot is rare, it should be fine to retry it.
yury-s
approved these changes
Sep 30, 2021
sidharthv96
added a commit
to sidharthv96/playwright
that referenced
this pull request
Oct 1, 2021
* upstream/master: (21 commits) fix(test runner): do not write missing snapshot until the last retry (microsoft#9246) feat(reporters): Add error position to JSON Report (microsoft#9151) feat(fetch): import/export storageState (microsoft#9244) fix: allow binary response interception (microsoft#9236) test(cookies): add a test for SameSite=None cookies (microsoft#9242) feat(webkit): roll WebKit to 1550 (microsoft#9239) test: add test for downloading PDF files (microsoft#9235) fix: leaking route handlers when times is used in Page.route (microsoft#9234) chore: upgrade commander.js to version 8 (microsoft#9230) feat(fetch): store cookies between requests (microsoft#9221) fix(selenium connect): register in gracefullyCloseAll for driver cleanup (microsoft#9218) fix(toBeHidden): return true to missing elements (microsoft#9205) chore: fix PlaywrightClient disconnection logic (microsoft#9149) fix(expect): beautiful expect stacks (microsoft#9204) feat(fetch): support ignoreHTTPSErrors option (microsoft#9206) feat(api): introduce locator.waitFor (microsoft#9200) feat(fetch): send Playwright as default user-agent for global fetch (microsoft#9195) feat(test runner): collect test error from worker teardown (microsoft#9190) test: add tests for Cross-Origin-Opener-Policy navigation (microsoft#9184) test: get response body for COOP responses (microsoft#9196) ...
sidharthv96
added a commit
to sidharthv96/playwright
that referenced
this pull request
Oct 1, 2021
…tionWithBaseChange * upstream/master: (21 commits) fix(test runner): do not write missing snapshot until the last retry (microsoft#9246) feat(reporters): Add error position to JSON Report (microsoft#9151) feat(fetch): import/export storageState (microsoft#9244) fix: allow binary response interception (microsoft#9236) test(cookies): add a test for SameSite=None cookies (microsoft#9242) feat(webkit): roll WebKit to 1550 (microsoft#9239) test: add test for downloading PDF files (microsoft#9235) fix: leaking route handlers when times is used in Page.route (microsoft#9234) chore: upgrade commander.js to version 8 (microsoft#9230) feat(fetch): store cookies between requests (microsoft#9221) fix(selenium connect): register in gracefullyCloseAll for driver cleanup (microsoft#9218) fix(toBeHidden): return true to missing elements (microsoft#9205) chore: fix PlaywrightClient disconnection logic (microsoft#9149) fix(expect): beautiful expect stacks (microsoft#9204) feat(fetch): support ignoreHTTPSErrors option (microsoft#9206) feat(api): introduce locator.waitFor (microsoft#9200) feat(fetch): send Playwright as default user-agent for global fetch (microsoft#9195) feat(test runner): collect test error from worker teardown (microsoft#9190) test: add tests for Cross-Origin-Opener-Policy navigation (microsoft#9184) test: get response body for COOP responses (microsoft#9196) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This prevents future retries from passing because of the actual snapshot being written.
In theory, we can avoid running the retry since it should fail anyway. However, this brings problems: for example running a test in
describe.serialmode has some global side effects and so it should not be skipped. Since running a test without a snapshot is rare, it should be fine to retry it.Fixes #9219.