Skip to content

Dismiss alert in UI tests to ensure tests pass the first time around#16118

Merged
guarani merged 1 commit intodevelopfrom
dismiss-alert-in-ui-tests
Mar 17, 2021
Merged

Dismiss alert in UI tests to ensure tests pass the first time around#16118
guarani merged 1 commit intodevelopfrom
dismiss-alert-in-ui-tests

Conversation

@guarani
Copy link
Contributor

@guarani guarani commented Mar 17, 2021

The new classic editor deprecation alert in Aztec (classic editor) was making all UI tests fail on the first try. CircleCI then ran tests a second time and because the alert is not shown again (state is persisted on the device), tests passed the second time around.

This was causing UI tests to fail the first time around for the last 10 days: https://app.circleci.com/pipelines/github/wordpress-mobile/WordPress-iOS?branch=develop. Since tests always passed the second time, this went unnoticed.

To test:

  1. Run optional UI tests (on any device)
  2. View the Run UI Tests job and check that tests pass the first time around (there should be no second attempt at running UI tests, i.e. no "Starting scan Improved QuickPicturePreviewView #2 with 10 tests.")

PR submission checklist:

  • I have considered adding unit tests where possible.
  • I have considered adding accessibility improvements for my changes.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

The new classic editor deprecation alert in Aztec (classic editor) is shown only once and was making all UI tests fail on the first try. CircleCI then ran tests a second time and because the alert is not shown again (a flag is persisted to the device to ensure this), tests passed the second time around.
@peril-wordpress-mobile
Copy link

You can trigger an installable build for these changes by visiting CircleCI here.

@guarani guarani added the Testing Unit and UI Tests and Tooling label Mar 17, 2021
@guarani guarani added this to the 17.0 milestone Mar 17, 2021
@peril-wordpress-mobile
Copy link

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

@guarani guarani requested a review from jkmassel March 17, 2021 22:04
Copy link
Contributor

@jkmassel jkmassel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done!

Thanks @guarani!!

@guarani guarani merged commit 72cedcc into develop Mar 17, 2021
@guarani guarani deleted the dismiss-alert-in-ui-tests branch March 17, 2021 23:12
@guarani
Copy link
Contributor Author

guarani commented Mar 17, 2021

Thanks @jkmassel!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Unit and UI Tests and Tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants