Skip to content

e2e CSV test - use unique name resources for each test#2541

Merged
openshift-merge-robot merged 1 commit intooperator-framework:masterfrom
akihikokuroda:e2ecsv
Jan 18, 2022
Merged

e2e CSV test - use unique name resources for each test#2541
openshift-merge-robot merged 1 commit intooperator-framework:masterfrom
akihikokuroda:e2ecsv

Conversation

@akihikokuroda
Copy link
Copy Markdown
Member

Signed-off-by: akihikokuroda akihikokuroda2020@gmail.com

The "ClusterServiceVersion when a CustomResourceDefinition was installed alongside a ClusterServiceVersion" tests uses the same namespace name for each test. It creates and remove the namespace every time. It doesn't delete the resources in the namespace before it delete the namespace. It may be causing the longer delete time of the namespace and the timeout of the test. The tests executed after the timeout fail because the uncleaned namespace and resources in it.

Description of the change:

  1. use the unique namespace, resource names for each test.
  2. delete the resources created in the test before the deletion of the namespace

Motivation for the change:
Closes #2540
Reviewer Checklist

  • Implementation matches the proposed design, or proposal is updated to match implementation
  • Sufficient unit test coverage
  • Sufficient end-to-end test coverage
  • Docs updated or added to /doc
  • Commit messages sensible and descriptive

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Dec 24, 2021

Hi @akihikokuroda. Thanks for your PR.

I'm waiting for a operator-framework member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 24, 2021
@akihikokuroda akihikokuroda changed the title use unique name for each test e2e CSV test - use unique name resources for each test Dec 24, 2021
Comment thread test/e2e/csv_e2e_test.go Outdated
Comment thread test/e2e/csv_e2e_test.go
Comment thread test/e2e/csv_e2e_test.go
}).Should(WithTransform(k8serrors.IsNotFound, BeTrue()))

Eventually(func() error {
return ctx.Ctx().Client().Delete(context.Background(), &ns)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this call is still prone to the previous behavior, but deleting the OG and CRD beforehand should smooth out the time that the namespace is hanging in the terminating status (on top of reducing the current flakiness of this test), and I don't have any better solutions poking around the code.

@timflannagan
Copy link
Copy Markdown
Member

@akihikokuroda Can you also rebase when you get a chance? I saw the unit test failure, but we just merged those changes since this PR has been opened.

@timflannagan
Copy link
Copy Markdown
Member

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 5, 2022
Comment thread test/e2e/csv_e2e_test.go Outdated
Comment thread test/e2e/csv_e2e_test.go Outdated
@timflannagan
Copy link
Copy Markdown
Member

@akihikokuroda You may need to also rebase against master too.

@awgreene
Copy link
Copy Markdown
Member

/approve

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 14, 2022
@akihikokuroda akihikokuroda force-pushed the e2ecsv branch 2 times, most recently from ad58d81 to 2600bc7 Compare January 18, 2022 15:57
@awgreene
Copy link
Copy Markdown
Member

@akihikokuroda could you rebase this test as well?

@akihikokuroda akihikokuroda force-pushed the e2ecsv branch 2 times, most recently from 41a99cf to 40ea2d4 Compare January 18, 2022 17:52
@timflannagan
Copy link
Copy Markdown
Member

@akihikokuroda Apologies - we just merged some e2e improvements that parallelize the e2e suite into their own jobs, so the overall e2e execution should drop from ~1h to ~20m. Can you rebase your PR once more, and I'll tag it once it passes the checks?

@timflannagan
Copy link
Copy Markdown
Member

Longer term, we need to solve the situation where branches constantly need to be rebased when changes land in the master branch as it's not the greatest UX for contributors.

Signed-off-by: akihikokuroda <akihikokuroda2020@gmail.com>
Copy link
Copy Markdown
Member

@timflannagan timflannagan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2022
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Jan 18, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akihikokuroda, awgreene, timflannagan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [awgreene,timflannagan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit e50fa36 into operator-framework:master Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e "ClusterServiceVersion when a CustomResourceDefinition was installed alongside a ClusterServiceVersion" test failures

4 participants