Changing catalog image source doesn't unpack again

[everettraven]

There seems to be a bug in the unpacking logic where the unpacking pod uses the same name as the catalog being unpacked. This results in a scenario where when you update the Catalog.Spec.Source.Image.Ref field to point to a new image reference it finds the unpacking pod used for a previous version of the Catalog and doesn't unpack the new reference.

A potential solution for this is to append the hash of the Catalog to the unpack pod name to ensure uniqueness.

[joelanford]

Rukpak re-uses the same pod name to unpack different bundle images, and I don't think reusing the same name causes problems. And it looks like catalogd uses pretty much identical logic.

In both cases, the controller attempts to apply the desired spec, and if applying the spec fails with an Invalid error, the controller deletes the pod and re-creates it with the desired spec.

[joelanford]

I'm seeing something unexpected. It looks like the kubernetes API server is accepting a change to the pod.spec.containers.image, but then nothing reconciles it. The status isn't changing at all.

[joelanford]

After some asking around, its clear that catalogd's expectation that bundle pods will be restarted when the catalog image changes is an invalid assumption.

This is not a problem in rukpak because the Bundle API is immutable.

So we need to do something to make sure the pod is restarted when the catalog image changes. Ideas include:

• always delete and recreate the pod if a patch would mutate the pod
• use a Job instead of a Pod
• switch entirely away from pod-based unpacking and instead pull directly from the image registry in the controller process.

[ncdc]

I vote to switch away from pods entirely. We still need to be cognizant of the amount of time it will take to unpack, so we don't tie up a worker unnecessarily long (we can always spawn some goroutines if needed). But I think this will be better.