Discussion: Use a Docker image for building binaries/files prior to generating SLSA provenances

[rbehjati]

Follow-up discussion about the idea of using a Docker image as the builder/releaser, as we have in project Oak.

In project Oak, and as part of our transparent-release work, we use a builder image for building binaries. The builder image is a Docker image, which has all the tools required for building the binary installed, and the required environment variables set. It might be interesting to use a similar idea here for building the binaries and generating the provenances. This can be used as an alternative to tools like go-releaser.

Currently in our tooling for transparent-release, the build command is a docker run command that runs a given command in the builder image. When generating SLSA provenances, we include this information as the BuildConfig. See also our custom buildType. In addition, we include the builder image in the list of materials. The build tool fetches the specified docker image and ensures that the command for building the binary is executed using the fetched builder image. The builder image is identified by a URI containing the digest of the image. If the versions of the toolchains are fixed in the Dockerfile (example from Oak) and the checksums are verified, then this can get very close to the idea of a trusted builder.

Here is an example of such a SLSA provenance file, with BuildConfig and materials as described above.

This is our GitHub action that generates provenances. We generate provenances for each commit that is merged into the main branch. It currently doesn’t use the build tool from transparent-release (because the build does not yet generate a provenance file), but we plan to use this build toll with a simple TOML file similar to this example. The idea is to have the TOML file checked into the repo as a static file (containing only the command, the output_path and a few other fields), and let the GitHub actions job fill out the commit_hash and the builder_image URI that are different for each commit and invocation of the build tool.

A similar minimal TOML can be used here for building the binary using a builder image provided by the maintainers of the repo.

Note that in our approach, we don’t fill out the invocation part in the SLSA predicate, as we think all the information is provided in the buildConfig, and materials.

cc @laurentsimon, @tiziano88

[laurentsimon]

Thanks for sharing @rbehjati !

I really like the approach of using docker to accommodate complex build systems. We could "easily" wrap this up in a reusable workflow to streamline the the work.

@MarkLodato @asraa wdut?

[asraa]

I really like the approach of using docker to accommodate complex build systems. We could "easily" wrap this up in a reusable workflow to streamline the the work.

+1!!!

I really like this -- currently this provenance generator is responsible for creating signed provenance populated with github context information. I was just chatting with Laurent on how we could easily create shared code that could apply to these docker image builds and other use cases on github workflows. For general use cases like yours:

(1) We can use the output of HostedActionProvenance

slsa-github-generator/slsa/provenance.go

Line 34 in 9a875d0

func HostedActionsProvenance(w WorkflowRun) (*intoto.ProvenanceStatement, error) {

to create a base statement with Github Workflow context information
(2) Provide library functions to augment with specific information like the buildConfig, custom buildType, and materials
(3) Generate and output signed provenances with either (A) Use Fulcio signer in this library if a raw output is needed or (B) cosign CLI signers in the case of uploading the provenance attached directly to the image (we can also raw output this).

In this tool the AI's for the above process would be:

• Support augmenting the provenance in library
• Output just the predicate json instead of the intoto statement. this would allow passing the predicate json to cosign

[asraa]

Are you planning on signing the provenance in order for the output of the trusted builder to be non-forgeable?

If so, where would you hold the provenance? Maybe on the provenance branch with a file commit like .sig? The .sig file would contain something like this, which contains the sig, certificate, and offline info to verify the transparency log information. Cosign understands this format as input.
https://gist.github.com/asraa/6471825cb23aaa053292348edcea0e2e

[ianlewis]

This looks pretty neat. I like the idea of having the build and provenance generation be more fully encapsulated.

If we can make sure that the context where the builder image gets executed is safe enough and can't alter the provenance down the line, we can maybe allow folks to specify their own image and the reusable workflow will be responsible for executing it and incorporating the information on the commands run into the provenance.

It seems worth exploring more.

[laurentsimon]

I also thought about users giving their own dockerfile: I was wondering if there are options (say, root docker) which may compromise security. But I think the consensus is that this is out of scope of SLSA, so I think it would work too.

[ianlewis]

Yeah, I from a technical standpoint we need to make sure that whatever does the build can't modify the provenance (i.e. command X was run but command Y was added to the provenance) but other than that we don't necessarily care what the build actually does).

I think we should be able to to things like get things like the entrypoint from the image? or just add the docker run command w/ image & image hash as a build step?

We probably do need a way to extract the build artifacts from the build job in a safe way too. Maybe the docker image would need to get the sha256sum of the artifacts and upload them itself? and then pass the names off to the provenance step somehow?

[tiziano88]

I think we should be able to to things like get things like the entrypoint from the image? or just add the docker run command w/ image & image hash as a build step?

We probably do need a way to extract the build artifacts from the build job in a safe way too. Maybe the docker image would need to get the sha256sum of the artifacts and upload them itself? and then pass the names off to the provenance step somehow?

I believe both these are covered by @rbehjati 's suggestion: the repo owner declares the Docker image digest (which is content-addressed, and needs to have been pre-built and pushed to some Docker registry) and the current git commit (which is also content-addressed) is mounted at a specific (possibly configurable) path under that docker image; also the owner specifies the command to run under this configuration, and what output file to measure; the measurement needs to be done in a trustworthy way by the workflow after the command has finished running (i.e. we don't need to trust the owner for this either).

These parameters may be provided to the workflow directly, or ideally as a separate TOML file. The latter has the advantage that we can build (and in fact have already built) tooling that parses it and runs the same exact steps locally, i.e. on the developer's machine, without having to rely on GitHub actions for instance while debugging things. Also it can be ported trivially to other CI systems; e.g. the same TOML file may be the input to the GitHub actions workflow, but also a Google Cloud Build workflow (without having to maintain two distinct workflow files in sync).

Finally a provenance file is created with all these pieces of information, plus whatever other metadata we get from the builder via the OIDC token.

[tiziano88]

Are you planning on signing the provenance in order for the output of the trusted builder to be non-forgeable?

If so, where would you hold the provenance? Maybe on the provenance branch with a file commit like .sig? The .sig file would contain something like this, which contains the sig, certificate, and offline info to verify the transparency log information. Cosign understands this format as input. https://gist.github.com/asraa/6471825cb23aaa053292348edcea0e2e

Another option that I would like to explore at some point is to set up an instance of https://github.com/google/ent (a universal Content Addressable Store) and store provenances there. It would be a nice counterpart to Rekor: Rekor stores signatures over hashes, Ent stores the actual artifacts (or provenances, or anything else really) indexed by their hash.

[rbehjati]

If we can make sure that the context where the builder image gets executed is safe enough and can't alter the provenance down the line, we can maybe allow folks to specify their own image and the reusable workflow will be responsible for executing it and incorporating the information on the commands run into the provenance.

That is exactly the idea with the build tool in transparent-release. It would be nice to further generalize this tool and turn it into a reusable library that can be used in GitHub workflows.

[ianlewis]

That is exactly the idea with the build tool in transparent-release. It would be nice to further generalize this tool and turn it into a reusable library that can be used in GitHub workflows.

Yeah, I'm sure we could do something similar and then to run the provenance generation step in a separate job from the builder as I'd not like to trust the builder job at least after the untrusted container gets executed.

It seems the path to the repo could be provided or you fetch it. I'm sure we could just checkout the repo in a build step, but I'm curious if there is any any benefit you saw to having the builder fetch the repo itself?
https://github.com/project-oak/transparent-release/blob/58721e709f89052ccbec8282543354f9a396dfdb/common/common.go#L351

[rbehjati]

It seems the path to the repo could be provided or you fetch it. I'm sure we could just checkout the repo in a build step, but I'm curious if there is any any benefit you saw to having the builder fetch the repo itself?

It is just for convenience. When running as a GitHub action, the repo is already checked out, so the option of passing the path to the builder would have to be used.

[laurentsimon]

cc @loosebazooka working on distroless - this may be the way we can generate provenance using a script/dockerfile

[ianlewis]

Just for clarity, there are two discussions we can have

• Allow folks to provide Dockerfile the workflow builds and/or pushes
• Allow folks to build an arbitrary artifact using a Docker image (this is what the Oak folks are proposing)

I think both have merit but probably need to be discussed separately. Probably implemented as different workflows.