Skip to content

Split UI Tests into Gmail vs Imap tests#437

Merged
tomholub merged 9 commits intomasterfrom
feature/issue-425-split-ui-tests
Aug 13, 2021
Merged

Split UI Tests into Gmail vs Imap tests#437
tomholub merged 9 commits intomasterfrom
feature/issue-425-split-ui-tests

Conversation

@Kharchevskyi
Copy link
Contributor

@Kharchevskyi Kharchevskyi commented Aug 11, 2021

This PR:

  • Added test plans for gmail and imap ui tests
  • Added script to run ui tests
  • Added UI tests for Imap login

close #425


Tests:

  • Added imap ui test for successful login.

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

  • addresses the issue it closes (if any)
  • code is readable and understandable
  • is accompanied with tests, or tests are not needed
  • is free of vulnerabilities
  • is documented clearly and usefully, or doesn't need documentation

Comment on lines +422 to +425
// Used for ui tests
if user.email.contains("default@flowcrypt.test") {
user.imap?.connectionType = "none"
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "none" here mean and what are the other possible values?

Copy link
Contributor Author

@Kharchevskyi Kharchevskyi Aug 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use providers.json to pre fill imap/smtp settings. Unfortunately it's not possible for mailcore lib to prefill with none value. Only starttls/ssl: true/false can be specified.

Simulator Screen Shot - iPhone 8 - 2021-08-11 at 20 06 23

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not set this through the UI in the test code? It's a UI test after all, it's expected to interact with UI elements during the tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this lines we will prefill with tls selected by default. (returned by default from mailcore).
If we change it after to none this will prefill with 143 port (default value from mailcore for none security type).
so then we need to change port as well both for imap and smtp.

This can be done on UI tests level, but this is a bit more time consuming and less stable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simulator Screen Shot - iPhone 8 - 2021-08-11 at 21 11 39

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand what you mean. Definitely we should use the UI for this. If it's less stable, we should over time find a way to improve the tests to make them more stable. It's possible for UI tests to be reasonably stable too, it just takes some work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I worried about that there are a lot of bugs inside xcode with ui tests.
For example, sometimes password text field should be tapped once to show keyboard and sometimes it works only from second tap.

let passwordTextField = app.tables.secureTextFields["Password"]
passwordTextField.tap()
passwordTextField.tap()

Will work on it(will mark as draft for now)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Hopefully we can work around all of this.

@Kharchevskyi Kharchevskyi marked this pull request as ready for review August 11, 2021 17:32
Copy link
Contributor Author

@Kharchevskyi Kharchevskyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@tomholub
Copy link
Collaborator

@Kharchevskyi please also update the PR template - the Tests section to reflect what was done in the PR. It's good to do this consistently to build a habit.

@tomholub tomholub changed the title Split UI Tests Split UI Tests into Gmail vs Imap tests Aug 11, 2021
@Kharchevskyi Kharchevskyi marked this pull request as draft August 11, 2021 18:23
@Kharchevskyi
Copy link
Contributor Author

lgtm

@Kharchevskyi Kharchevskyi marked this pull request as ready for review August 13, 2021 14:03
Copy link
Collaborator

@tomholub tomholub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tomholub
Copy link
Collaborator

please look at #439 next

@tomholub tomholub merged commit e79c2b7 into master Aug 13, 2021
@tomholub tomholub deleted the feature/issue-425-split-ui-tests branch August 13, 2021 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

split tests into Gmail tests and IMAP tests

2 participants