Added Several `Contact Support` tests. by pachlava · Pull Request #15744 · wordpress-mobile/WordPress-Android

pachlava · 2021-12-22T10:01:04Z

Description

Since the ability to contact support is considered one of the most critical, this PR adds several tests:

helpCanBeOpenedWhileEnteringPassword: checks that Help button is accessible from the password prompt screen.
helpCanBeOpenedWhileEnteringPassword: checks that Help button is accessible from the email prompt screen.
sendButtonEnabledWhenTextIsEntered: checks that Send button switches states correctly depending on the entered message text.
messageCanBeSent (disabled, only executable with gradle.properties from Mobile Secrets, more here): sends a message and checks that a certain reply is returned.

Testing Instructions

The Connected Tests-2 CI job is green, and you can see helpCanBeOpenedWhileEnteringPassword, sendButtonEnabledWhenTextIsEntered , helpCanBeOpenedWhileEnteringEmail tests included.
Associated Firebase execution is also green (Firebase console access is needed to check this).
Tests run locally (both with default gradle.properties and the ones from Mobile Secrets) on Pixel 2 API 28 (this is the model used on CI).
[optional] Enable and run messageCanBeSent locally with gradle.properties from Mobile Secrets.

Regression Notes

Potential unintended areas of impact:
All changes took place in the package related to automated tests. The app behaviour should not be affected.
What I did to test those areas of impact (or what existing automated tests I relied on):
I executed all existing automated tests, and the CI is green.
What automated tests I added:
I added:

sendButtonEnabledWhenTextIsEntered
messageCanBeSent
helpCanBeOpenedWhileEnteringEmail
helpCanBeOpenedWhileEnteringPassword

PR submission checklist:

I have completed the Regression Notes.
I have considered adding accessibility improvements for my changes.
I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

peril-wordpress-mobile · 2021-12-22T10:01:08Z

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

peril-wordpress-mobile · 2021-12-22T10:15:17Z

You can test the changes on this Pull Request by downloading the APKs:

Jetpack-pr-15744-build-117887.apk

WordPress-pr-15744-build-117887.apk

This reverts commit 08fd20d.

pachlava · 2021-12-23T19:26:54Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/pages/ContactSupportScreen.java

+    static ViewInteraction textInput = onView(allOf(
+            isCompletelyDisplayed(),
+            anyOf(
+                    withId(R.id.message_composer_input_text),
+                    withId(R.id.request_message_field)
+            )
+    ));


This translates to: find an element that is both completely displayed and has either R.id.message_composer_input_text or R.id.request_message_field ID.

In fact, both of these elements are present when the screen is opened, but only one is visible.

pachlava · 2021-12-23T19:29:32Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/ContactUsTests.java

+        }
+    }
+
+    @Ignore("As long as CI does not use gradle.properties from MobileSecrets")


As can be seen from the number of commits in this PR starting with CI fail fix:..., I spent quite a while figuring out why the tests that involve Contact Support screen failed on CI but never failed the same way locally. At some point (thanks to video recording in Firebase) I saw that the screen on CI:

Looks different from the same screen on my local Emulator:

After blaming the Emulator/Android differences for some time, I realized that CI uses the default gradle.properties, which makes the screen look like differently (and the message cannot be sent, hence the disabled test).

Since I automated this test, I decided to keep it for the case if Mobile Secrets will be used on CI.

In case we have this test running in CI are we going to be creating real Zendesk tickets? I'm afraid it can get real noisy.

I have this suspicion too. This test is disabled for now, so it's not an urgent matter, but I think we can contact the HE team, and ask them if there's a painless way for them to filter out certain messages, so that they wouldn't be bothered with them.

Yeah, I agree.

If it turns out to be the case that we cannot prevent the creation of real tickets, and idea could be to use the Zendesk APIs to find the tickets created by the UI tests and delete them.

pachlava · 2021-12-23T19:31:02Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/ContactUsTests.java

+        } finally {
+            new ContactSupportScreen().goBackAndDeleteUnsentMessageIfNeeded();
+        }


A kind of local teardown, in order to not slow down helpCanBeOpenedWhileEnteringEmail and helpCanBeOpenedWhileEnteringPassword which do not get to ContactSupport screen, so there's no possibility of unsent message.

pachlava · 2021-12-23T19:38:40Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/pages/HelpAndSupportScreen.java

+        onView(anyOf(
+                withText(android.R.string.ok),
+                withId(android.R.id.button1)
+        ))


This is the locator for OK button in this dialog:

I used something like onView(withText("OK")) initially, and this worked locally, but never on CI. Thanks to this discussion, I found out that "OK" button might be presented differently by different Android versions, while android.R.id.button1 always stands for the "positive" option in the dialog.

tiagomar · 2021-12-27T19:55:24Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/ContactUsTests.java

+        String automatedReplyText = "Mobile support will respond as soon as possible, "
+                                    + "generally within 48-96 hours. "
+                                    + "Please reply with your site address (URL) "
+                                    + "and any additional details we should know.";


I have this test failing since the message have slightly changed due to holidays. We might want to do a partial check as the two last lines remained the same.

Yes. So far, I'm not sure if it makes sense to maintain this test (or even to keep it) - I probably firstly need to ask if Zendesk connection is ever planned on CI.

We have it connected for WPiOS because it was not possible to even check if the screen had been loaded. From what I can recall it is possible but not desirable. Given it is an identified critical flow I'd ask @jkmassel to weigh in. =)

We could also use a less sophisticated check and ensure we get any text in R.id.request_system_message_text. 🤷‍♂️

tiagomar · 2021-12-27T20:19:03Z

WordPress/src/androidTest/java/org/wordpress/android/support/WPSupportUtils.java

+    public static boolean waitForElementToBeDisplayedWithoutFailure(final ViewInteraction element) {
+        try {
+            waitForConditionToBeTrueWithoutFailure(new Supplier<Boolean>() {
+                @Override
+                public Boolean get() {
+                    return isElementDisplayed(element);
+                }
+            });
+        } catch (Exception e) {
+            // ignore the failure
+        }
+        return isElementDisplayed(element);
+    }
+


I had to do the same thing in my PR. The only difference was that I made waitForElementToBeDisplayedWithoutFailure(final Integer elementID) call waitForElementToBeDisplayedWithoutFailure(final ViewInteraction element) to avoid a bit of code duplication.
Nothing to do here.

tiagomar

CI is green, I ran the tests locally several times and the only issue I found was the Zendesk response message which seems to change seasonally, however this test is disabled so no impact is expected in CI and it will no longer be a problem the next week. That said, I feel comfortable to approve this PR and have this test fixed later on in another PR.

mokagio

Thank you for putting this together!

mokagio · 2021-12-31T05:30:40Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/ContactUsTests.java

+        }
+    }
+
+    @Ignore("As long as CI does not use gradle.properties from MobileSecrets")


If it turns out to be the case that we cannot prevent the creation of real tickets, and idea could be to use the Zendesk APIs to find the tickets created by the UI tests and delete them.

mokagio · 2021-12-31T05:32:09Z

WordPress/src/androidTest/java/org/wordpress/android/e2e/ContactUsTests.java

+        String automatedReplyText = "Mobile support will respond as soon as possible, "
+                                    + "generally within 48-96 hours. "
+                                    + "Please reply with your site address (URL) "
+                                    + "and any additional details we should know.";


We could also use a less sophisticated check and ensure we get any text in R.id.request_system_message_text. 🤷‍♂️

tiagomar · 2022-01-03T20:24:34Z

Since @pachlava will be AFK this week and the PR has two approvals already, I'll merge it now.

pachlava · 2022-01-10T10:42:19Z

Thanks for the review @tiagomar (and for the merge while I was AFK) and @mokagio!

Added Several Contact Support tests.

0f69f25

pachlava marked this pull request as draft December 22, 2021 10:01

pachlava added 11 commits December 22, 2021 12:29

Adjusted the imports to fix checkstyle warnings

a57a010

Variables renaming to solve stylecheck warnings

d4e294b

CI fail fix: added a conditional wait before entering an email.

68c5a3a

CI fail fix: added more robust conditional wait when entering email

b492893

CI fail fix: Using a different locator for OK button

2539abd

CI fail fix: Rewrote a part that sets user email address and name.

5153c27

CI fail fix: added one more conditional wait

77025df

CI fail fix: finally figured out the reason (fingerscrossed)

08fd20d

Revert "CI fail fix: finally figured out the reason (fingerscrossed)"

e176a68

This reverts commit 08fd20d.

CI fail fix: finally figured out the reason (hopefully)

7c93b54

Updated a comment.

11a16e7

pachlava added the UI Tests Anything related to automated UI Tests. label Dec 23, 2021

pachlava added this to the Future milestone Dec 23, 2021

pachlava commented Dec 23, 2021

View reviewed changes

pachlava marked this pull request as ready for review December 23, 2021 19:41

pachlava requested review from jostnes, mokagio and tiagomar December 23, 2021 19:41

pachlava changed the title ~~[Draft] Added Several Contact Support tests.~~ Added Several Contact Support tests. Dec 23, 2021

tiagomar reviewed Dec 27, 2021

View reviewed changes

tiagomar approved these changes Dec 27, 2021

View reviewed changes

mokagio approved these changes Dec 31, 2021

View reviewed changes

tiagomar merged commit bdcad4c into develop Jan 3, 2022

tiagomar deleted the adds-contact-us-UI-test branch January 3, 2022 20:24

tiagomar mentioned this pull request Jan 4, 2022

remove duplicated WPSupportUtils method #15789

Merged

3 tasks

jostnes mentioned this pull request Oct 6, 2022

[UI Tests] - Re-enable disabled UI Tests #17281

Closed

9 tasks

Conversation

pachlava commented Dec 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing Instructions

Regression Notes

Uh oh!

peril-wordpress-mobile bot commented Dec 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

peril-wordpress-mobile bot commented Dec 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pachlava Dec 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tiagomar left a comment

Choose a reason for hiding this comment

Uh oh!

mokagio left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tiagomar commented Jan 3, 2022

Uh oh!

pachlava commented Jan 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pachlava commented Dec 22, 2021 •

edited

Loading

peril-wordpress-mobile bot commented Dec 22, 2021 •

edited

Loading

peril-wordpress-mobile bot commented Dec 22, 2021 •

edited

Loading

pachlava Dec 23, 2021 •

edited

Loading