Skip to content

adds the derived image test scripts for prow #769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 1 commit into from
Apr 6, 2022
Merged

adds the derived image test scripts for prow #769

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions ci/derived-image-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/bin/bash

# This script performs the following tasks:
#
# 1. Logs into the rhcos-devel service account to get the image push credential
# so we can push to the registry.ci.openshift.org/rhcos-devel namespace.
#
# 2. Kicks off the build-test-qemu.sh script to build and test an RHCOS image
# with Kola tests.
#
# 3. Pushes the resulting image to the image registry using the Prow build ID
# as the tag since we need to have the image in this registry so our derived
# image build test can use it.
#
# 3. Kicks off the derived OS image testing binary which pulls the newly built
# image, builds a derived image from it, applies it to an underlying OpenShift
# cluster node, verifies that it was successfully applied, then rolls the node
# back.

set -euo pipefail

# Note: The oc binary will be injected by the Prow CI process. It
# is not present in ci/Dockerfile.
oc login https://api.ci.l2s4.p1.openshiftapps.com:6443 --token="$(cat /service-account-token/image-pusher-service-account-token)"
oc registry login --registry=registry.ci.openshift.org --to="$SHARED_DIR/dockercfg.json"

export COSA_DIR="/tmp/cosa"
mkdir -p "$COSA_DIR"

# Run the cosa build / test
/src/ci/build-test-qemu.sh

export REGISTRY_AUTH_FILE="$SHARED_DIR/dockercfg.json"

# Ensure we're in the designated cosa directory so the push-container commands work
cd "$COSA_DIR"

# Tags with the cosa build ID / arch - unique to this specific build
Copy link
Copy Markdown
Member

@cgwalters cgwalters Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will anything prune these tags by default? From a quick look at openshift/release I see image pruners set up on the CI clusters but not the central CI registry?

Copy link
Copy Markdown
Member Author

@cheesesashimi cheesesashimi Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to be honest. It would be nice to push to the ephemeral CI registry first so one doesn't have to worry about pruning these tags.

cosa push-container registry.ci.openshift.org/rhcos-devel/rhel-coreos

# Tag with the Prow Build ID because we don't want to overwrite our well-known
# tags yet, but our test cluster needs the image to be pushed someplace so we
# can ingest it. We use the BUILD_ID value because its unique to each job so
Copy link
Copy Markdown
Member

@cgwalters cgwalters Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Doesn't need to be in this PR but I think we should add a TODO here about how we actually do want to push to the "CI namespace" that prow creates that is naturally lifecycle bound to the PR, but we just can't do that right now because Prow wants to own the build process)

Copy link
Copy Markdown
Member Author

@cheesesashimi cheesesashimi Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about that and only pushing to the external registry on success since that would eliminate the need to prune the build ID tags. However, there are two things I need to figure out:

  1. How to get those creds within a test since the KUBECONFIG env var points to ones ephemeral test cluster, not the ephemeral CI registry?
  2. Is the ephemeral test cluster is created with the perms to pull from the CI namespace registry? I have to assume that it is, but I could be wrong.

# they won't stomp on each other if running concurrently.
#
# TODO: Aim to push this to the ephemeral CI namespace registry before making
# the final push at the end.
export BASE_IMAGE_PULLSPEC="registry.ci.openshift.org/rhcos-devel/rhel-coreos:$BUILD_ID"
cosa push-container "$BASE_IMAGE_PULLSPEC"

# Perform the derived OS image build tests
/usr/local/bin/layering_test -test.v -test.failfast -test.timeout 35m -build-log="$ARTIFACT_DIR/derived-image-build.log"
26 changes: 26 additions & 0 deletions ci/tag-final-image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash

# This script performs the following tasks:
#
# 1. Reads the jobspec to get the active branch so we know what tag
# to use.
#
# 2. Tags the image tagged with the Prow build ID to one of our
# well-known tags (e.g., master, 4.11, latest, etc.)

set -euo pipefail

# We can't use PULL_BASE_REF or OPENSHIFT_BUILD_REFERENCE to get the
# branch since this is a periodic job which originates from
# openshift/release, not the openshift/os repository. We then strip
# release- from the branch name so we're left with the number (e.g.,
# release-4.11 -> 4.11).
BRANCH="$(echo "$JOB_SPEC" | jq -r '.extra_refs[0].base_ref | sub("release-"; "")')"
export REGISTRY_AUTH_FILE="$SHARED_DIR/dockercfg.json"

skopeo copy "docker://registry.ci.openshift.org/rhcos-devel/rhel-coreos:$BUILD_ID" "docker://registry.ci.openshift.org/rhcos-devel/rhel-coreos:$BRANCH"

# Only push latest tag on master branch
if [[ "$BRANCH" == "master" ]]; then
skopeo copy "docker://registry.ci.openshift.org/rhcos-devel/rhel-coreos:$BUILD_ID" "docker://registry.ci.openshift.org/rhcos-devel/rhel-coreos:latest"
fi
12 changes: 3 additions & 9 deletions tests/layering/build_helper_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,6 @@ import (
"k8s.io/apimachinery/pkg/util/wait"
)

// Gets the RHCOS base image tag from the BASE_IMAGE_TAG environment variable.
func getBaseImageTag() string {
return getEnvVarOrDefault("BASE_IMAGE_TAG", "latest")
}

// Gets the value of an environment variable or defaults to the provided
// default.
func getEnvVarOrDefault(envVarName, defaultValue string) string {
Expand Down Expand Up @@ -108,13 +103,12 @@ func (b *builder) createImageStream(ctx context.Context) (*imagev1.ImageStream,

// Actually perform the OS derivation build and waits for it to complete.
func (b *builder) buildDerivedOSImage(ctx context.Context) error {
baseImageBuildArg := fmt.Sprintf(imagePullSpec, getBaseImageTag())

baseImagePullSpec := getEnvVarOrDefault("BASE_IMAGE_PULLSPEC", "registry.ci.openshift.org/rhcos-devel/rhel-coreos:latest")
derivationRepoURL := getEnvVarOrDefault("DERIVATION_REPO_URL", "https://github.com/coreos/fcos-derivation-example")
derivationRepoRef := getEnvVarOrDefault("DERIVATION_REPO_REF", "rhcos")
dockerfilePath := getEnvVarOrDefault("DERIVATION_DOCKERFILE_PATH", "Dockerfile")

b.t.Log("base image pullspec:", baseImageBuildArg)
b.t.Log("base image pullspec:", baseImagePullSpec)
b.t.Log("derivation repo URL:", derivationRepoURL)
b.t.Log("derivation repo ref:", derivationRepoRef)
b.t.Log("dockerfile path:", dockerfilePath)
Expand All @@ -139,7 +133,7 @@ func (b *builder) buildDerivedOSImage(ctx context.Context) error {
BuildArgs: []corev1.EnvVar{
{
Name: "RHEL_COREOS_IMAGE",
Value: baseImageBuildArg,
Value: baseImagePullSpec,
},
},
},
Expand Down
1 change: 0 additions & 1 deletion tests/layering/layering_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ const (

buildName = imageStreamName
helloWorldPath = "/usr/bin/hello-world"
imagePullSpec = "registry.ci.openshift.org/rhcos-devel/rhel-coreos:%s"
imageRegistry = "image-registry.openshift-image-registry.svc:5000"
imageStreamName = "test-boot-in-cluster-image"
imageURL = ostreeUnverifiedRegistry + ":" + imageRegistry + "/" + mcoNamespace + "/" + imageStreamName
Expand Down