EnsureImage should validate content hash, not just URL presence

[tonicmuroq]

Problem

EnsureImage in vm/cocoon_cli.go:69-82 checks whether the base image is available locally by running cocoon image inspect <image>. If inspect succeeds (image name exists in the local index), it returns immediately without pulling.

func (c *CocoonCLI) EnsureImage(ctx context.Context, image string) error {
    if c.command(ctx, "image", "inspect", image).Run() == nil {
        return nil // name exists → skip
    }
    // pull only if name not found
    c.command(ctx, "image", "pull", image)
}

This means if the upstream image behind a mutable tag (:latest) is replaced with a new build, EnsureImage still considers the old cached version sufficient. The snapshot's COW overlay then references a backing file blob that doesn't match the locally cached blob, and cocoon vm clone fails:

Backing file I/O error: /var/lib/cocoon/cloudimg/blobs/<expected-hash>.qcow2
No such file or directory (os error 2)

Where it matters

provider/cocoon/create.go:130:

if snapshot != nil && snapshot.Image != "" {
    p.Runtime.EnsureImage(ctx, snapshot.Image)
}

This runs before every clone to ensure the base image is present. But "present" currently means "any version of this URL is cached", not "the version this snapshot was built against is cached".

Proposed fix

Option A (ideal): The snapshot manifest already records the base image reference. If the snapshot also stored the base image's content digest (the blob SHA256), EnsureImage could compare it against the locally cached blob and re-pull on mismatch.

Option B (pragmatic): Respect an imagePullPolicy annotation on the pod (similar to Kubernetes' Always / IfNotPresent / Never). When set to Always, EnsureImage would delete + re-pull unconditionally.

Option C (minimal, depends on cocoonstack/cocoon#18): If cocoon image pull --force is implemented upstream, EnsureImage could use it when the snapshot's base image reference doesn't match the cached content digest.

Workaround

Manually run cocoon image rm <url> && cocoon image pull <url> on each cocoonset node before cloning from a snapshot that was built against a newer base image.

Related

• cocoon image pull: add --force flag to bypass URL-level cache cocoon#18 — cocoon image pull --force flag

[CMGS]

#3 merged