SBOM attestations generation

[jedevc]

Follow up to #2935, and part 2 of #2773 (comment).

This PR adds support for scanning the resulting image to easily generate SBOMs. For example, using my example scanner at jedevc/buildkit-syft-scanner:

$ buildctl ... build --frontend=dockerfile.v0 --output type=image,name=<image>,push=true --opt attest=sbom=jedevc/buildkit-syft-scanner:latest

This generates SPDX attestations according to the In Toto SPDX attestation.

The jedevc/buildkit-syft-scanner image follows a simple scanning protocol:

• The default cmd should contain the scanning command
• On run, the image should scan the root filesystem located at BUILDKIT_SCAN_SOURCE, and output an SPDX document to BUILDKIT_SCAN_DESTINATION.

The details of this protocol will likely change, however, we need to ensure that anyone interested in producing an SBOM scanner can easily create an image to slot into the build process. Ideally, we'd also be able to provide a default, however, we should do this as a later follow-up, once the overall design is stabilized.

The scanner is integrated into the dockerfile frontend natively, and, if a frontend does not produce an SBOM when specified, the solver will fallback to this same scanner.

Still to-do:

• Write a test suite, using a basic testing scanner.
• Determine the user interface. We will aim to support an --attest flag for buildx, however, how this option is passed to buildkit (part of the frontend attributes, or part of the exporter attributes) is up for debate.
• Define the minimal scanning protocol (documentation for the protocol should be added to our docs)

Note to reviewers: this PR shares many commits with #2935, see the comparison here to see only commits that are different.

[jedevc]

To test locally, you can use justinchadwell050/buildkit:sbom-attestations and justinchadwell050/dockerfile:sbom-attestations:

$ docker buildx create --driver docker-container --name sbom-builder --driver-opt image=justinchadwell050/buildkit:sbom-attestations
$ docker buildx build --driver sbom-builder --build-arg BUILDKIT_SYNTAX=justinchadwell050/dockerfile:sbom-attestations ...

[jedevc]

The one major remaining thing still missing from this is the ability to bundle attestations internally, which will allow the sbom scanning protocol to output sboms in different formats, which will be picked up by the exporter.

[tonistiigi]

Looking at my local branch, this PR used to have some dockerfile tests. Did something get deleted?

[jedevc]

Have pushed functionality to do attestation bundling to let a generator produce multiple outputs per input. This requires a new version of jedevc/buildkit-syft-scanner.

[jedevc]

Have enhanced the scanning protocol to clarify how multiple refs are scanned at a time. A scan has one "primary" ref to scan (the runtime rootfs), but may contain multiple "extra" refs which could be used in the future to attach SBOMs of build contexts or of previous build stages.

Otherwise, I'm reasonably happy with the state of this, though we'd need a follow-up to introduce documentation for creating custom scanners.

[jedevc]

Have rebased, should be ready to review/merge.

I did draft a new commit to maybe pull on top of this: d4ced27 (with the corresponding change jedevc/buildkit-syft-scanner@e6fa1e1).

This would change the structure of the bundle fairly significantly - but the idea would be that instead of having an index.json that contains all the attestation metadata, each individual file in the bundle would be a full in-toto statement, so we no longer need a metadata file - this means we could use directory traversal to extract all the files in the bundle. I'd like to have this instead of a index file, since it's easier to integrate with (buildkit-syft-scanner is a lot simpler) - additionally, it would make building a dockerfile syntax with attestation mounts as in #2773 (comment).

[tonistiigi]

wdyt of controlling the extras here with defining ARG SBOM_SCAN_STAGE=true and global ARG SBOM_SCAN_CONTEXT=true (in addition to allowing it to be set in attestation attrs).

[jedevc]

So a stage that wants to be a scan point has a command for that ARG? That sounds ok, but I think it would be good to expose some level of control to the user somehow.

Maybe:

• mode=min = scan only the final stage, regardless of ARGs
• mode=max = scan the final stage, and also all the scan points (that are declared on the ARGs), maybe with a warning if no additional scan points have been point?

I'm not sure, what we would do if a user overrode those ARGs using build --arg? I think the user's choice is taken into account and we override the values in the Dockerfile.

Let's do this in a follow-up ↪️