pulling with --storage ostree doesn't enforce signatures

[aweiteka]

When running atomic pull with ostree storage backend signature policy is not enforced because the skopeo layers command doesn't enforce signatures (see containers/skopeo#287). Is there a reason we're not using skopeo copy here? https://github.com/projectatomic/atomic/blob/master/Atomic/syscontainers.py#L1172

/cc @giuseppe

[rhatdan]

I don't think skopeo supports an ostree backend? @runcom

[runcom]

@rhatdan it doesn't, it relies on a-now-deprecated command called "skopeo layers" to import individual layer to an ostree repo on a system.

[rhatdan]

Right so to get skopeo copy to work we would need to have a destination like ostree:/var/lib/containers
and put the code to install the layers into skopeo correct?

[runcom]

Yeah, that would be nice, @giuseppe what do you think? I remember some conversion with @giuseppe and @mtrmac somewhere but I couldn't find the link anywhere right now.

[mtrmac]

A low-effort fix might be to get containers/image#159 + a corresponding skopeo enhancement, and use skopeo copy --layer-subset instead of skopeo layers. That would allow us to preserve the existing Python code, at the cost of having to be really careful so that we always do verification, and only pull exactly one image (I suppose, first pull the manifest + no layers, and then use digest references, using a digest computed from that manifest, for all subsequent operations).

Alternatively we could build an ostree ImageTransport directly into containers/image in Go (and drop the Python code from atomic?).

[rhatdan]

I would prefer solution #2
we could build an ostree ImageTransport directly into containers/image in Go (and drop the Python code from atomic?).

[runcom]

The key functionality we need to port from python to go in containers/image is the ability to check for already pulled layers to do incremental pulls on image update (@giuseppe can tell us more about it I guess). But with the changes in c/image such as HasBlob we should be able to maintain that compatibility.

[cgwalters]

I'd at some point like to be able to pull OCI from ostree, or at least rpm-ostree. See discussion in ostreedev/ostree#619

[rhatdan]

That would be very cool

[mtrmac]

containers/image does have a (manifest) format conversion machinery which could be useful for this, even if images did not originate as OCI images (or the other way around).

[giuseppe]

From my understanding of containers/skopeo#242, skopeo copy doesn't allow us to specify what layers to download yet. That is a blocker for atomic pull --storage ostree, as we want to fetch only the layers that are not already in the repository (shouldn't this be done for Docker as well?).

atomic pull --storage ostree fetches the manifest, lookup the layers of the image and then request to skopeo layers only those that are not already in the OSTree repository.

[aweiteka]

It appears skopeo copy pulls down manifest schema v2 (if available) and skopeo layers pulls down manifest schema v1.

[mtrmac]

It appears skopeo copy pulls down manifest schema v2 (if available) and skopeo layers pulls down manifest schema v1.

True; and actually unnecessary nowadays AFAICT, we can parse the v2 data as well; it seems to just be a leftover implementation artifact.

[mtrmac]

Really the question here is whether we should decide to port all of this to Go, or whether to keep the current code and only to implement containers/image#159 and make it available in skopeo. (Though we now have a PR to implement something like that anyway, containers/image#218, so that may happen whichever way we decide to take.)