doc: add base doc that clarifies how users can test their projects#3823
Merged
camilamacedo86 merged 3 commits intooperator-framework:masterfrom Oct 14, 2020
camilamacedo86:doc-tests
Merged
doc: add base doc that clarifies how users can test their projects#3823camilamacedo86 merged 3 commits intooperator-framework:masterfrom camilamacedo86:doc-tests
camilamacedo86 merged 3 commits intooperator-framework:masterfrom
camilamacedo86:doc-tests
Conversation
camilamacedo86
requested review from
bharathi-tenneti,
dmesser,
jmccormick2001,
jmrodri and
varshaprasad96
and removed request for
hasbro17 and
shawn-hurley
camilamacedo86
commented
Sep 1, 2020
jberkhahn
reviewed
Sep 23, 2020
jberkhahn
reviewed
Sep 23, 2020
jberkhahn
reviewed
Sep 23, 2020
jberkhahn
reviewed
Sep 23, 2020
jberkhahn
reviewed
Sep 24, 2020
jberkhahn
reviewed
Sep 24, 2020
Contributor
|
Couple more nits, then this looks good to go. |
Contributor
Author
|
Hi @jberkhahn, Really thank you for your review. All nits addressed. |
jberkhahn
reviewed
Sep 25, 2020
| go test controllers/ -v -ginkgo.v | ||
| ``` | ||
|
|
||
| The projects generated by using the SDK tool have a Makefile which contains the target tests which executes when you run make test. Note that this target will also execute when you run `make docker-build IMG=<some-registry>/<project-name>:<tag>` the test also will be executed. |
Contributor
There was a problem hiding this comment.
the last bit of the sentence "the test also will be executed" is uneeded and can be deleted.
Contributor
|
one more thing and this although your tests seem to be blowing up |
camilamacedo86
commented
Sep 30, 2020
Contributor
|
/lgtm |
openshift-ci-robot
added
lgtm
Contributor
|
/lgtm |
jmrodri
changed the title
jmrodri
requested changes
Oct 7, 2020
Member
jmrodri
left a comment
jmrodri
left a comment
There was a problem hiding this comment.
Great start, some minor cleanup. Once that's done I'll lgtm
| ## Migrate your tests | ||
|
|
||
| For the new layout, you will see that `controllers/suite_test.go` is created. This file contains boilerplate for executing integration tests using [envtest][envtest] with [ginkgo](https://onsi.github.io/ginkgo/) and [gomega](https://onsi.github.io/gomega/). | ||
| For the new layout, you will see that `controllers/suite_test.go` is created when a controller is scaffolded by the tool. This file contains boilerplate for executing integration tests using [envtest][envtest] with [ginkgo](https://onsi.github.io/ginkgo/) and [gomega](https://onsi.github.io/gomega/). |
Member
There was a problem hiding this comment.
I guess we should just be consistent with how we handle links. Add ginkgo and gomega to the bottom with the other links.
| Operator SDK 1.0.0+ removes support for the legacy test framework and no longer supports the `operator-sdk test` subcommand. All affected tests should be migrated to use `envtest`. | ||
|
|
||
| The Operator SDK project recommends controller-runtime's [envtest][envtest] because it has a more active contributor community, it has become more mature than Operator SDK's test framework, and it does not require an actual cluster to run tests, which can be a huge benefit in CI scenarios. | ||
| The Operator SDK project recommends using controller-runtime's [envtest][envtest] to write tests for your Operators projects because it has a more active contributor community, it has become more mature than Operator SDK's test framework, and it does not require an actual cluster to run tests, which can be a huge benefit in CI scenarios. |
Member
There was a problem hiding this comment.
Split the sentence up.
The Operator SDK project recommends using controller-runtime's [envtest][envtest] to write tests for your Operators projects. Envtest has a more active contributor community, it is more mature than Operator SDK's test framework, and it does not require an actual cluster to run tests which can be a huge benefit in CI scenarios.
|
|
||
| ## Overview | ||
|
|
||
| The Operator SDK project recommends using controller-runtime's [envtest][envtest] to write tests for your Operators projects because it has a more active contributor community, it has become more mature than Operator SDK's test framework, and it does not require an actual cluster to run tests, which can be a huge benefit in CI scenarios. |
Member
There was a problem hiding this comment.
Split the sentence up.
The Operator SDK project recommends using controller-runtime's [envtest][envtest] to write tests for your Operators projects. Envtest has a more active contributor community, it is more mature than Operator SDK's test framework, and it does not require an actual cluster to run tests which can be a huge benefit in CI scenarios.
|
|
||
| ## Using EnvTest | ||
|
|
||
| You will see that `controllers/suite_test.go` is created when a controller is scaffolded by the tool. This file contains boilerplate for executing integration tests using [envtest][envtest] with [ginkgo](https://onsi.github.io/ginkgo/) and [gomega](https://onsi.github.io/gomega/). |
Member
There was a problem hiding this comment.
We should just be consistent with how we handle links. Add ginkgo and gomega to the bottom with the other links.
| go test controllers/ -v -ginkgo.v | ||
| ``` | ||
|
|
||
| The projects generated by using the SDK tool have a Makefile which contains the target tests which executes when you run make test. Note that this target will also execute when you run `make docker-build IMG=<some-registry>/<project-name>:<tag>`. |
|
|
||
| ## Other Options | ||
|
|
||
| Also you can write tests for your operator in a declarative using the [kuttl](https://kuttl.dev/). Via kuttl, you can define YAML manifests that specify the expected before and after statues of a cluster when your operator is used. For more info see [Writing Kuttl Scorecard Tests][writing-kuttl-scorecard-tests]. |
Member
There was a problem hiding this comment.
Also, you can write tests for your operator in a declarative manner using [kuttl][kuttl]. With kuttl, you define YAML manifests that contain the expected before and after states of the cluster. For more info see [Writing Kuttl Scorecard Tests][writing-kuttl-scorecard-tests].
Put the kuttl link with the other links in the doc.
kuttl
|
|
||
| Also you can write tests for your operator in a declarative using the [kuttl](https://kuttl.dev/). Via kuttl, you can define YAML manifests that specify the expected before and after statues of a cluster when your operator is used. For more info see [Writing Kuttl Scorecard Tests][writing-kuttl-scorecard-tests]. | ||
|
|
||
| To implement application-specific tests, the test harness of the SDK, [scorecard][scorecard], provides the ability to ship custom code in container images as well, which can be referenced in the test suite. Because this test suite definition metadata travels with the Operator Bundle, it allows for functional testing of the Operator without the source code or the project layout being available. See [Writing Custom Scorecard Tests][writing-custom-scorecard-tests]. |
Member
There was a problem hiding this comment.
change: the test harness of the SDK -> the SDK's test harness
| [envtest-setup]: /docs/building-operators/golang/references/envtest-setup | ||
| [writing-kuttl-scorecard-tests]: /docs/advanced-topics/scorecard/kuttl-tests | ||
| [writing-custom-scorecard-tests]: /docs/advanced-topics/scorecard/custom-tests | ||
| [scorecard]: /docs/advanced-topics/scorecard/scorecard No newline at end of file |
Contributor
Author
|
Really thank you for your time @jmrodri and help with this review. |
estroz
pushed a commit
to estroz/operator-sdk
that referenced
this pull request
Oct 14, 2020
…perator-framework#3823) **Description of the change:** doc: add base doc to clarifies how users can test their projects **Motivation for the change:** Many users have raised these questions. The goal of this doc is to provide the basic information and the links/references for they are able to move forward. It can be improved by the community and/or in the post 1.0 doc plans. Closes: operator-framework#3511
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of the change:
doc: add base doc to clarifies how users can test their projects
Motivation for the change:
Many users have raised these questions. The goal of this doc is to provide the basic information and the links/references for they are able to move forward. It can be improved by the community and/or in the post 1.0 doc plans.
Closes: #3511
Checklist
If the pull request includes user-facing changes, extra documentation is required:
changelog/fragments(seechangelog/fragments/00-template.yaml)website/content/en/docs