[e2e-test] Add run bundle to e2e test

[varshaprasad96]

Description of the change:
Add run bundle to e2e operator test suite.

Motivation for the change:
Test run bundle integration with OLM.

Checklist

If the pull request includes user-facing changes, extra documentation is required:

• Add a new changelog fragment in changelog/fragments (see changelog/fragments/00-template.yaml)
• Add or update relevant sections of the docs website in website/content/en/docs

[varshaprasad96]

Ginko's skip works only for blocks within Its. Hence this needs to be on hold till run bundle is implemented, or could be commented and merged.
Note: Various run bundle scenarios will be tested in olm_integration_suite.

[camilamacedo86]

See my comment:

• instead of use docker-push we load the image in the kind. Use LoadImageToKindClusterWithName(bundleImage)
• The same need to be done for helm/ansible as well.

[camilamacedo86]

Suggested change

	Enable `run bundle` command and include e2e tests.
	Add the subcommand `operato-sdk run bundle` which allows users check their project integration with OLM via the bundle.

The e2e are not relevant to end-users

[jmrodri]

I agree with @camilamacedo86 rephrase it. I would change her suggestion a bit:

Added the run bundle subcommand which allows users to verify their operator's bundle integrates with OLM.

[estroz]

This should be sufficient:

Suggested change

	Enable `run bundle` command and include e2e tests.
	Added the `run bundle` command.

[camilamacedo86]

Suggested change

// TODO(joelanford): enable bundle command when implementation is complete

[camilamacedo86]

You are running the bundle in a specific namespace. The ns that. is created for the operator (project-system).
Should not the cleanup be done with the ns as well?

[camilamacedo86]

The command is not working see:

See that the error lead to us to beleive that the image was NOT load in the Kind. So, the problem shows be in the isRunningOnKind. It shows has not the kubectx outside of the suite. See;

operator-sdk/test/e2e-ansible/e2e_ansible_suite_test.go

Lines 196 to 198 in f7d9976

	func isRunningOnKind() bool {
	return strings.Contains(kubectx, "kind")
	}

. You can debug the tests locally to check it.

Suggested solution:

• Move the var kubectx in the suite to the testContext:

// TestContext wraps kubebuilder's e2e TestContext.
type TestContext struct {
	*kbtestutils.TestContext

         // Kubectx stores the k8s context from where the tests are running
         Kubectx string
}

• Move isRunningOnKind to utils. See;

  operator-sdk/test/internal/utils.go

  Line 89 in f7d9976

  func (tc TestContext) LoadImageToKindClusterWithName(image string) error {

func (tc TestContext) IsRunningOnKind() bool {
      return strings.Contains(tc.Kubectx, "kind")
}

• the if will be tc.IsRunningOnKind on the tests
• In the suite code we will set the value: tc.Kubectx = the type

[varshaprasad96]

Yes, was trying to debug this error with Jesus. IsRunningOnKind works fine, but the run bundle logic always reaches to the remote registry to pull and unpack the image. Jesus is working on finding a way out for that. Meanwhile we decided to push the image to a remote repository, and check if it works. But there is some error in the install plan. Trying to figure that out.

[camilamacedo86]

Hi @varshaprasad96,

Since we use the IsRunningOnKind in all tests and in more than one place, IMO it should be centralized such as in order to be easy keep it maintained: #3941

Regards the issue, see that in the above pr;

STEP: loading the bundle image into Kind cluster
running: kind load docker-image e2e-test/controller-manager-bundle:mbnd --name kind
STEP: running the operator bundle using run bundle command
running: operator-sdk run bundle e2e-test/controller-manager-bundle:mbnd --namespace e2e-mbnd-system

Which means that the image was loaded in the Kind. So, the problem shows be a bug in the operator-sdk run bundle.

[varshaprasad96]

@camilamacedo86 yes, there is no error with IsRunningOnKind. The implementation logic of run bundle expects the image to be in registry. Even if its loaded in the kind cluster, while unpacking it tries to pull it from registry.

[jmrodri]

Just minor changes to the fragment and etc.

[jmrodri]

I agree with @camilamacedo86 rephrase it. I would change her suggestion a bit:

Added the run bundle subcommand which allows users to verify their operator's bundle integrates with OLM.

[estroz]

I assume you're reverting this change @varshaprasad96?

[jmrodri]

@estroz no, afaik the image has to be pushed otherwise run bundle can't seem to pull the code.

[camilamacedo86]

Hi @varshaprasad96,

It is outdated and we need to solve the issue with the run bundle to be able to do that?
Do you think is valid keep this PR open or track an issue and link this one for we have the history to know how we can do the new PR? So, if we move forward in this way, would be nice to have a link for the issue to solve the run bundle issue as well.

WDYT @jmrodri

[varshaprasad96]

@camilamacedo86 this PR can be closed. These test cases would be covered in run bundle unit tests.