-
Notifications
You must be signed in to change notification settings - Fork 1.8k
[run-bundle] Test example scenarios for run bundle #3844
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,235 @@ | ||||||||||||||||||
| // Copyright 2020 The Operator-SDK Authors | ||||||||||||||||||
| // | ||||||||||||||||||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||||||||||||||||||
| // you may not use this file except in compliance with the License. | ||||||||||||||||||
| // You may obtain a copy of the License at | ||||||||||||||||||
| // | ||||||||||||||||||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||||||||||||||||||
| // | ||||||||||||||||||
| // Unless required by applicable law or agreed to in writing, software | ||||||||||||||||||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||||||||||||||||||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||||||||||||||
| // See the License for the specific language governing permissions and | ||||||||||||||||||
| // limitations under the License. | ||||||||||||||||||
|
|
||||||||||||||||||
| // Modified from https://github.com/kubernetes-sigs/kubebuilder/tree/39224f0/test/e2e/v3 | ||||||||||||||||||
|
|
||||||||||||||||||
| package e2e | ||||||||||||||||||
|
|
||||||||||||||||||
| import ( | ||||||||||||||||||
| "context" | ||||||||||||||||||
| "fmt" | ||||||||||||||||||
| "path" | ||||||||||||||||||
| "path/filepath" | ||||||||||||||||||
| "strings" | ||||||||||||||||||
| "testing" | ||||||||||||||||||
| "time" | ||||||||||||||||||
|
|
||||||||||||||||||
| . "github.com/onsi/ginkgo" | ||||||||||||||||||
| . "github.com/onsi/gomega" | ||||||||||||||||||
| "github.com/operator-framework/api/pkg/operators/v1alpha1" | ||||||||||||||||||
| "github.com/operator-framework/operator-sdk/internal/olm/operator" | ||||||||||||||||||
| "github.com/operator-framework/operator-sdk/internal/olm/operator/bundle" | ||||||||||||||||||
| testutils "github.com/operator-framework/operator-sdk/test/internal" | ||||||||||||||||||
| "github.com/sirupsen/logrus" | ||||||||||||||||||
| corev1 "k8s.io/api/core/v1" | ||||||||||||||||||
| wait "k8s.io/apimachinery/pkg/util/wait" | ||||||||||||||||||
| "sigs.k8s.io/controller-runtime/pkg/client" | ||||||||||||||||||
| kbtestutils "sigs.k8s.io/kubebuilder/test/e2e/utils" | ||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the imports need to be sorted. |
||||||||||||||||||
| ) | ||||||||||||||||||
|
|
||||||||||||||||||
| // TODO: After run bundle is complete, remove the boolean variable and the if statements. | ||||||||||||||||||
| var runBundleImplemented = false | ||||||||||||||||||
|
|
||||||||||||||||||
| var ( | ||||||||||||||||||
| tc testutils.TestContext | ||||||||||||||||||
| err error | ||||||||||||||||||
| ) | ||||||||||||||||||
|
|
||||||||||||||||||
| var _ = BeforeSuite(func() { | ||||||||||||||||||
| By("creating a new context") | ||||||||||||||||||
| // For this test suite, the bundle image name is equal to tc.ImageName | ||||||||||||||||||
| tc, err = testutils.NewTestContext("GO111MODULE=on") | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
| Expect(tc.Prepare()).To(Succeed()) | ||||||||||||||||||
|
|
||||||||||||||||||
| createOperator(tc, defaultOperatorName) | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| var _ = AfterSuite(func() { | ||||||||||||||||||
| By("removing container image and working dir") | ||||||||||||||||||
| tc.Destroy() | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| var _ = Describe("run bundle", func() { | ||||||||||||||||||
| It("Run Bundle Basic", func() { | ||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||
| if !runBundleImplemented { | ||||||||||||||||||
| Skip("Run bundle not yet implemented") | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| cfg := &operator.Configuration{KubeconfigPath: kubeconfigPath} | ||||||||||||||||||
| err := cfg.Load() | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
Comment on lines
+70
to
+72
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why it is rerquired?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These are a part of |
||||||||||||||||||
|
|
||||||||||||||||||
| i := bundle.NewInstall(cfg) | ||||||||||||||||||
| i.BundleImage = tc.ImageName | ||||||||||||||||||
|
Comment on lines
+74
to
+75
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are running the make bundle for he image. what the NewInstall does? Why it is required?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The bundle image is an argument to the run bundle command. When the command is passed, the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why not we do not use operator-sdk run bundle here? IMO : It is an e2e test so we should use the CLI commands otherwise we could cover these scenarios with unit tests. |
||||||||||||||||||
|
|
||||||||||||||||||
| err = doInstall(i) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
Comment on lines
+77
to
+78
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What the doinstall does?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same above. we need to use the CLI command instead. |
||||||||||||||||||
|
|
||||||||||||||||||
| uninstall(kubeconfigPath, tc) | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| It("Run bundle SingleNamespace", func() { | ||||||||||||||||||
| if !runBundleImplemented { | ||||||||||||||||||
| Skip("Run bundle not yet implemented") | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| cfg := &operator.Configuration{KubeconfigPath: kubeconfigPath, Namespace: tc.Kubectl.Namespace} | ||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is an e2e test and we are creating the project. So, we need to test it as users would be performing this action. Have we a flag to define the run bundle is single namespace or it users need to update the manifest generated? |
||||||||||||||||||
| err := cfg.Load() | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| i := bundle.NewInstall(cfg) | ||||||||||||||||||
| i.BundleImage = tc.ImageName | ||||||||||||||||||
| i.InstallMode.InstallModeType = v1alpha1.InstallModeTypeSingleNamespace | ||||||||||||||||||
| i.InstallMode.TargetNamespaces = []string{tc.Kubectl.Namespace} | ||||||||||||||||||
|
|
||||||||||||||||||
| err = doInstall(i) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| uninstall(kubeconfigPath, tc) | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| It("Run bundle OwnNamespace", func() { | ||||||||||||||||||
| if !runBundleImplemented { | ||||||||||||||||||
| Skip("Run bundle not yet implemented") | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| cfg := &operator.Configuration{KubeconfigPath: kubeconfigPath} | ||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This may not be testing |
||||||||||||||||||
| err := cfg.Load() | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| i := bundle.NewInstall(cfg) | ||||||||||||||||||
| i.BundleImage = tc.ImageName | ||||||||||||||||||
| i.InstallMode.InstallModeType = v1alpha1.InstallModeTypeOwnNamespace | ||||||||||||||||||
| i.InstallMode.TargetNamespaces = []string{tc.Kubectl.Namespace} | ||||||||||||||||||
|
|
||||||||||||||||||
| err = doInstall(i) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| uninstall(kubeconfigPath, tc) | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| It("Run Bundle - watching the specified namesapce", func() { | ||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What do you mean by "watching the specified namespace"? This case is testing |
||||||||||||||||||
| if !runBundleImplemented { | ||||||||||||||||||
| Skip("Run bundle not yet implemented") | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| cfg := &operator.Configuration{KubeconfigPath: kubeconfigPath, Namespace: tc.Kubectl.Namespace} | ||||||||||||||||||
| err := cfg.Load() | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| By("create a namespace to watch") | ||||||||||||||||||
| _, err = tc.Kubectl.Command("create", "namespace", "bar") | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| i := bundle.NewInstall(cfg) | ||||||||||||||||||
| i.BundleImage = tc.ImageName | ||||||||||||||||||
| i.InstallMode.InstallModeType = v1alpha1.InstallModeTypeSingleNamespace | ||||||||||||||||||
| i.InstallMode.TargetNamespaces = []string{tc.Kubectl.Namespace, "bar"} | ||||||||||||||||||
|
|
||||||||||||||||||
| err = doInstall(i) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| uninstall(kubeconfigPath, tc) | ||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| }) | ||||||||||||||||||
|
|
||||||||||||||||||
| // createOperator scaffolds a new test operator, generates bundle and pushes it to a remote | ||||||||||||||||||
| // repository. The generated csv by default supports OwnNamespace, SingleNamespace and AllNamespaces. | ||||||||||||||||||
| func createOperator(tc testutils.TestContext, projectName string) { | ||||||||||||||||||
| By("initializing a project") | ||||||||||||||||||
| err := tc.Init( | ||||||||||||||||||
| "--project-version", "3-alpha", | ||||||||||||||||||
| "--repo", path.Join("github.com", "example", projectName), | ||||||||||||||||||
| "--domain", tc.Domain, | ||||||||||||||||||
| "--fetch-deps=false") | ||||||||||||||||||
| Expect(err).Should(Succeed()) | ||||||||||||||||||
|
|
||||||||||||||||||
| By("creating an API definition") | ||||||||||||||||||
| err = tc.CreateAPI( | ||||||||||||||||||
| "--group", tc.Group, | ||||||||||||||||||
| "--version", tc.Version, | ||||||||||||||||||
| "--kind", tc.Kind, | ||||||||||||||||||
| "--namespaced", | ||||||||||||||||||
| "--resource", | ||||||||||||||||||
| "--controller", | ||||||||||||||||||
| "--make=false") | ||||||||||||||||||
| Expect(err).Should(Succeed()) | ||||||||||||||||||
|
|
||||||||||||||||||
| By("implementing the API") | ||||||||||||||||||
| Expect(kbtestutils.InsertCode( | ||||||||||||||||||
| filepath.Join(tc.Dir, "api", tc.Version, fmt.Sprintf("%s_types.go", strings.ToLower(tc.Kind))), | ||||||||||||||||||
| fmt.Sprintf(`type %sSpec struct { | ||||||||||||||||||
| `, tc.Kind), | ||||||||||||||||||
| ` // +optional | ||||||||||||||||||
| Count int `+"`"+`json:"count,omitempty"`+"`"+` | ||||||||||||||||||
| `)).Should(Succeed()) | ||||||||||||||||||
|
Comment on lines
+171
to
+178
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It shows not required for the test at all. Why we need to add an value in the types? Could we not use the foo? |
||||||||||||||||||
|
|
||||||||||||||||||
| By("generating the operator bundle") | ||||||||||||||||||
| // Turn off interactive prompts for all generation tasks. | ||||||||||||||||||
| replace := "operator-sdk generate kustomize manifests" | ||||||||||||||||||
| testutils.ReplaceInFile(filepath.Join(tc.Dir, "Makefile"), replace, replace+" --interactive=false") | ||||||||||||||||||
| err = tc.Make("bundle", "IMG="+tc.ImageName) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| By("building the operator bundle image") | ||||||||||||||||||
| // Use the existing image tag but with a "-bundle" suffix. | ||||||||||||||||||
| err = tc.Make("bundle-build", "BUNDLE_IMG="+tc.ImageName) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| By("push the bundle to a remote repository") | ||||||||||||||||||
| err = tc.Make("docker-push", "IMG="+tc.ImageName) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
Comment on lines
+192
to
+194
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
we should not use the docker-push here. The tests are executed using Kind. We build the images and then we load them. See: |
||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| func uninstall(kubeconfig string, tc testutils.TestContext) { | ||||||||||||||||||
| cfg := &operator.Configuration{KubeconfigPath: kubeconfig} | ||||||||||||||||||
| err := cfg.Load() | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| uninstallConfig := operator.NewUninstall(cfg) | ||||||||||||||||||
| uninstallConfig.DeleteAll = true | ||||||||||||||||||
| uninstallConfig.DeleteOperatorGroupNames = []string{tc.Group} | ||||||||||||||||||
| uninstallConfig.Package = defaultOperatorName | ||||||||||||||||||
| uninstallConfig.Logf = logrus.Infof | ||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what should be uninstalled here? Is the bundle?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, it is the bundle. As you said, after creating a new test context for the entire suite, before each test we
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will not the tool provide a command to uninstall? |
||||||||||||||||||
|
|
||||||||||||||||||
| ctx, cancel := context.WithTimeout(context.Background(), defaultTimeout) | ||||||||||||||||||
| defer cancel() | ||||||||||||||||||
|
|
||||||||||||||||||
| err = uninstallConfig.Run(ctx) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
|
|
||||||||||||||||||
| waitForConfigMapDeletion(ctx, cfg, defaultOperatorName) | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| func waitForConfigMapDeletion(ctx context.Context, cfg *operator.Configuration, packageName string) { | ||||||||||||||||||
| cfgmaps := corev1.ConfigMapList{} | ||||||||||||||||||
| opts := []client.ListOption{ | ||||||||||||||||||
| client.InNamespace(cfg.Namespace), | ||||||||||||||||||
| client.MatchingLabels{"owner": "operator-sdk", "package-name": packageName}, | ||||||||||||||||||
| } | ||||||||||||||||||
| err := wait.PollImmediateUntil(250*time.Millisecond, func() (bool, error) { | ||||||||||||||||||
| if err := cfg.Client.List(ctx, &cfgmaps, opts...); err != nil { | ||||||||||||||||||
| return false, err | ||||||||||||||||||
| } | ||||||||||||||||||
| return len(cfgmaps.Items) == 0, nil | ||||||||||||||||||
| }, ctx.Done()) | ||||||||||||||||||
| Expect(err).NotTo(HaveOccurred()) | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| func TestRunBundle(t *testing.T) { | ||||||||||||||||||
| RegisterFailHandler(Fail) | ||||||||||||||||||
| RunSpecs(t, "Operator SDK run bundle test suite") | ||||||||||||||||||
| } | ||||||||||||||||||
|
Comment on lines
+232
to
+235
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. in order to follow up a standard each dir should have 1 suite test and the tests that will be called by it. IMO we need re-check if is really required to keep this dir.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As discussed with @jmrodri and @estroz, this PR was structured to follow the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK. See that :
So, this e2e test would just make sense if we would like to cover all possible scenarios. Is this the goal? If yes, OK. However, note that So, if we will add tests to cover all options of the
Comment on lines
+232
to
+235
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All integration tests in this package should be converted to ginkgo/gomega tests eventually like this integration test is, so the suite runner should be in a separate file and look like so:
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMO we need to create the dir /test/e2e-olm/ in order to follow the project standard. The tests here are old and are not following the new definitions. Then, we should run |
||||||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// should be the name of the dir with _test
Assuming that it will be in /tests/e2e_olm as suggested: