Automate Testing/Validation of Gallery Component Pages

[chiaramooney]

Currently when we need to validate a release, we update gallery to the latest RNW version and then manually walk through the pages to confirm that all of the components are working and rendering correctly. Instead, it would be great to automate this process by using some kind of UI testing platform - such as Jest - to iterate through the pages and test the controls. We should then add this testing to our pipelines, so we can catch regressions as they come.

[chrisglein]

Goal would be to update the "test updated gallery app" checklist item from here with something that mostly runs itself.

[chrisglein]

Update from @chiaramooney: we have snapshot testing for gallery and upgrades/builds against canary, latest, and preview but we are still missing interactive E2E tests

[chiaramooney]

Next goal is to get interactive E2E testing done. We will need to do some initial setup here to get WebDriverIO/WinAppDriver setup and flow of navigating to a particular component that we want to test (est. ~half a week of work). From there it should take us 1-2 days per component we are trying to add interactive testing for.

[chrisglein]

Next goal is to get interactive E2E testing done. We will need to do some initial setup here to get WebDriverIO/WinAppDriver setup and flow of navigating to a particular component that we want to test (est. ~half a week of work). From there it should take us 1-2 days per component we are trying to add interactive testing for.

There are ~30 components for Gallery today, so that could be 30-60 days based on this estimate. Can you suggest a prioritization for what would be the best return? For example a P1/P2/P3 list. So maybe we have ~3 days of initial setup, then we go after 10 components for a total of ~20 days of investment for much better test coverage.