Skip to content

Add Dev Catalog e2e tests to Jenkins CI #1082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
openshift-merge-robot merged 1 commit into from
Jan 29, 2019
Merged

Add Dev Catalog e2e tests to Jenkins CI #1082

openshift-merge-robot merged 1 commit into from
Jan 29, 2019

Conversation

@dtaylor113
Copy link
Contributor

@dtaylor113 dtaylor113 commented Jan 15, 2019 •
edited
Loading

  • added test for when no service classes loaded in Dev Catalog
  • updated catalog-page to update when clusterServiceVersions change
  • renamed catalog.scenario.ts to developer-catalog.scenario.ts and added it to serviceCatalog test suite, which is run during Jenkins CI

@openshift-ci-robot openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 15, 2019
spadgett
spadgett reviewed Jan 15, 2019
View reviewed changes
Copy link
Member

@spadgett spadgett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dtaylor113

frontend/integration-tests/protractor.conf.ts Outdated
Copy link
Member

@spadgett spadgett Jan 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest just making this part of the serviceCatalog suite, particularly if it requires service catalog enabled to pass.

Copy link
Contributor Author

@dtaylor113 dtaylor113 Jan 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@dtaylor113 dtaylor113 force-pushed the dev-catalog-e2e-updates branch from 5b675e3 to 31b93d2 Compare January 15, 2019 18:22
@dtaylor113 dtaylor113 changed the title Add OperatorHub and Dev Catalog e2e tests to Jenkins CI Add Dev Catalog e2e tests to Jenkins CI Jan 15, 2019
spadgett
spadgett reviewed Jan 15, 2019
View reviewed changes
@dtaylor113 dtaylor113 force-pushed the dev-catalog-e2e-updates branch from 31b93d2 to 363f87c Compare January 16, 2019 16:27
spadgett
spadgett approved these changes Jan 16, 2019
View reviewed changes
Copy link
Member

@spadgett spadgett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dtaylor113

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 16, 2019
@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 16, 2019

/retest

@openshift openshift deleted a comment from spadgett Jan 17, 2019
@openshift openshift deleted a comment from spadgett Jan 17, 2019
@openshift openshift deleted a comment from spadgett Jan 17, 2019
@openshift openshift deleted a comment from spadgett Jan 17, 2019
@openshift openshift deleted a comment from spadgett Jan 17, 2019
@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 17, 2019

Trying to verify Service Catalog e2e test w/ dev catalog passes locally, shared cluster seems to be unstable

@spadgett
Copy link
Member

spadgett commented Jan 17, 2019

This error can happen periodically when looping over DOM nodes in tests

1) Catalog : filters catalog tiles by 'Service Class' Type correctly
   StaleElementReferenceError: stale element reference: element is not attached to the page document
     (Session info: headless chrome=67.0.3396.62)

@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 17, 2019

This error can happen periodically when looping over DOM nodes in tests

I know, it's on lines like: expect(catalogPageView.catalogTileFor('.NET Core').isDisplayed()).toBe(true); which are also on the operator hub e2e tests and have worked for a while now. Verified the css selector hasn't changed. Still investigating....

@spadgett
Copy link
Member

spadgett commented Jan 17, 2019

Yeah, it won't happen every time. It will just flake occasionally.

which are also on the operator hub e2e tests and have worked for a while now.

Those tests aren't currently gating merges, so it's hard to say how often they flake.

@spadgett
Copy link
Member

spadgett commented Jan 17, 2019

jenkins rebuild

3 similar comments
@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

jenkins rebuild

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

jenkins rebuild

@jhadvig
Copy link
Member

jhadvig commented Jan 18, 2019

jenkins rebuild

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

Last failure was flake https://jira.coreos.com/browse/CONSOLE-1214

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

jenkins rebuild

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

Last failed on flake https://jira.coreos.com/browse/CONSOLE-1219

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

jenkins rebuild

@spadgett
Copy link
Member

spadgett commented Jan 18, 2019

/hold

Holding for now due to the StaleElementReferenceError. I think we should try to figure that out before merging, particularly since our queue has recently been flakey.

We should determine why the catalog is re-rendering so frequently, but it'd be good to get away from using filter() if possible as well.

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 18, 2019
@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 18, 2019

We should determine why the catalog is re-rendering so frequently

https://jira.coreos.com/browse/CONSOLE-1223

@dtaylor113 dtaylor113 force-pushed the dev-catalog-e2e-updates branch from 363f87c to 1f15451 Compare January 18, 2019 20:04
@openshift-ci-robot openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 18, 2019
@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 22, 2019

jenkins rebuild

@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 22, 2019

@spadgett This PR seems ready to go! -thanks

@dtaylor113 dtaylor113 force-pushed the dev-catalog-e2e-updates branch from 1f15451 to e02ba69 Compare January 23, 2019 14:43
spadgett
spadgett approved these changes Jan 24, 2019
View reviewed changes
Copy link
Member

@spadgett spadgett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

There's some follow on work required to enable service catalog in the console-e2e tests so we can continue to gate merges on the service catalog scenarios. The Jenkins tests are going away soon.

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 24, 2019
@spadgett
Copy link
Member

spadgett commented Jan 24, 2019

jenkins rebuild

1 similar comment
@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 25, 2019

jenkins rebuild

@dtaylor113
Copy link
Contributor Author

dtaylor113 commented Jan 25, 2019

e2e failure:

Failed Specs:

1) Visiting Overview page : shows an emtpy list when no resources exist
   Error: Failed expectation
       at Object.<anonymous> /go/src/github.com/openshift/console/frontend/integration-tests/tests/overview/overview.scenario.ts:25:65
       at step /go/src/github.com/openshift/console/frontend/integration-tests/tests/overview/overview.scenario.ts:32:23
       at Object.next /go/src/github.com/openshift/console/frontend/integration-tests/tests/overview/overview.scenario.ts:13:53
       at fulfilled /go/src/github.com/openshift/console/frontend/integration-tests/tests/overview/overview.scenario.ts:4:58
Level { name_: 'SEVERE', value_: 1000 } 'http://localhost:9000/path-ZHAFIxCrKH/api/kubernetes/api/v1/namespaces/openshift-logging/configmaps/sharing-config - Failed to load resource: the server responded with a status of 404 (Not Found)'

@spadgett
Copy link
Member

spadgett commented Jan 25, 2019

jenkins rebuild

FYI, this isn't needed anymore (and will start some unnecessary tests)

@spadgett
Copy link
Member

spadgett commented Jan 25, 2019

/hold cancel

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 25, 2019
@spadgett
Copy link
Member

spadgett commented Jan 25, 2019

e2e failed because the install failed

/retest

@spadgett
Copy link
Member

spadgett commented Jan 29, 2019

/retest

@openshift openshift deleted a comment from spadgett Jan 29, 2019
@openshift openshift deleted a comment from spadgett Jan 29, 2019
@openshift-merge-robot openshift-merge-robot merged commit af38aff into openshift:master Jan 29, 2019
@dtaylor113 dtaylor113 deleted the dev-catalog-e2e-updates branch August 22, 2019 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spadgett spadgett spadgett approved these changes

@jeff-phillips-18 jeff-phillips-18 jeff-phillips-18 approved these changes

Assignees

@spadgett spadgett

Labels

lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@dtaylor113 @spadgett @jhadvig @jeff-phillips-18 @openshift-ci-robot @openshift-merge-robot