Generalize container-push to support multi-arch images and varying artifacts

[rmohr]

The container-push command now works like this:

• It checks the meta.json file of every built architecture referenced in builds.json
• It creates with buildah a multi-arch image and pushes it.
• It determines the final digest values for the remove manifests per arch (to be independend of the push format) and updates meta.json of each architecture.

Right now the command works with ostree and kubevirt. If not arthifact is specified it falls back to ostree to not interrupt any build pipelines wich assume that it solely works for ostree.

Pushing ostree:

cosa push-container quay.io/myorg/ostree:testing --authfile=/home/builder/.docker/config.json --format oci ostree

Push kubevirt:

cosa push-container quay.io/myorg/kubevirt:testing --authfile=/home/builder/.docker/config.json --format oci kubevirt

Note: Since this modifie meta.json stream metadata needs to be regenerated after all artifacts are pushed.

Background discussion happened at coreos/fedora-coreos-pipeline#515 (comment).

[cgwalters]

I think this is at least the 3rd version of this we have in tree now...there's also class Builds etc.

[cgwalters]

We also have this bit in the mantle code, but fine as is.

[cgwalters]

Thanks for working on this BTW!

[openshift-ci]

@rmohr: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cgwalters]

I am uncertain about this PR...it ties into the bigger multiarch discussion.

What I don't quite understand is this command seems to assume the containers have already been built for multiple architectures and stored in meta.json, and that the local build has fetched all those architectures.

I think we should more clearly separate "multi-arch unification" flows from the "individual build command".

The "multi-arch unification" (i.e. convert arch-specific container builds into a standard manifest list, or convert arch-specific cosa disk image builds into stream metadata) I think should be a separate command.

I pushed some (textually but not logically) conflicting code in #2907 FWIW.

[ravanelli]

What I don't quite understand is this command seems to assume the containers have already been built for multiple architectures and stored in meta.json, and that the local build has fetched all those architectures.

Yes, in theory gangplank would allow us to have all multi-arch artifacts and the unified meta.json in the same place. You wouldn't need to fetch/build it as part of container-push

Since we are probably dropping gangplank, we may think if that's what we really want, have everything unified in the sam e place? If so, it could be some enhancement for podman-remote part @dustymabe is starting.

For the manifest part, we really don't need to have the containers in the host itself. We can push them directly from the remote to a registry and then create a manifest pointing to it.

We can still change this PR to look for both cases.

[rmohr]

Closing this one since I am not actively working on it. Doing some cleanup. Thanks everyone for your help.