Skip to content

dockerfile: add testReproSourceDateEpoch#3456

Merged
AkihiroSuda merged 1 commit intomoby:masterfrom
AkihiroSuda:testSourceDateEpochRepro
Jan 24, 2023
Merged

dockerfile: add testReproSourceDateEpoch#3456
AkihiroSuda merged 1 commit intomoby:masterfrom
AkihiroSuda:testSourceDateEpochRepro

Conversation

@AkihiroSuda
Copy link
Copy Markdown
Member

Tests bit-for-bit reproducibility of OCI tar

Comment thread frontend/dockerfile/dockerfile_test.go Outdated
@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch from 9eccafa to 9c5512a Compare January 5, 2023 07:56
@AkihiroSuda AkihiroSuda changed the title dockerfile: add testSourceDateEpochRepro dockerfile: add testReproSourceDateEpoch Jan 5, 2023
@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch 3 times, most recently from 41dc466 to 0c0c7a7 Compare January 5, 2023 15:47
@AkihiroSuda AkihiroSuda marked this pull request as ready for review January 5, 2023 15:51
@AkihiroSuda AkihiroSuda marked this pull request as draft January 5, 2023 16:13
Comment thread frontend/dockerfile/dockerfile_test.go
@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch from 0c0c7a7 to 9803c9f Compare January 5, 2023 17:27
@AkihiroSuda AkihiroSuda marked this pull request as ready for review January 5, 2023 23:05
@AkihiroSuda
Copy link
Copy Markdown
Member Author

@tonistiigi Can we merge this?
We should make the digest independent of the snapshotter, but I guess that can be another PR.

Comment thread frontend/dockerfile/dockerfile_test.go Outdated
t.Logf("SOURCE_DATE_EPOCH=%d", tm.Unix())

dockerfile := []byte(`# The base image cannot be busybox, due to https://github.com/moby/buildkit/issues/3455
FROM --platform=linux/amd64 debian:bullseye-20221219-slim@sha256:171530d298096f0697da36b3324182e872db77c66452b85783ea893680cc1b62
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alpine doesn't work either? If possible would like to avoid another non-mirrored image.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine's /bin/find lacks -newermt, -writable, etc.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we can't find a better solution, we might still need to add it to mirrors. Otherwise I think even different workers and frontend types in build matrix will each do their own pull.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine's /bin/find lacks -newermt, -writable, etc.

apk add findutils ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine's /bin/find lacks -newermt, -writable, etc.

apk add findutils ?

That will result in an unreproducible build, and pinning the apk version is hard

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That will result in an unreproducible build, and pinning the apk version is hard

Correct. https://github.com/tonistiigi/buildkit-alpine

Does integration.WithMirroredImages in L227 suffice?

But that is not with a digest? Also pin it directly to amd64-only digest if the test is amd64 specific.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to amd64-only digest

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't pass in arm anymore 😢

    run.go:158: 
                Error Trace:    /src/frontend/dockerfile/run.go:158
                                                        /src/frontend/dockerfile/dockerfile_test.go:228
                Error:          Received unexpected error:
                                docker.io/arm64v8/debian:bullseye-20230109-slim@sha256:1acb06a0c31fb467eb8327ad361f1091ab265e0bf26d452dea45dcb0c0ea5e75: not found
                Test:           TestIntegration

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you put the pins in https://github.com/moby/buildkit/blob/master/util/testutil/integration/pins.go (and use amd64 digest for arm, with a comment) then I think it should work.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch 2 times, most recently from ad671f1 to 4524803 Compare January 13, 2023 06:55
@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch from 4524803 to 1f76bc6 Compare January 19, 2023 05:18
@AkihiroSuda
Copy link
Copy Markdown
Member Author

@tonistiigi PTAL 🙏

@tonistiigi
Copy link
Copy Markdown
Member

@AkihiroSuda I still had an issue on arm64 with the latest version but haven't had a chance to debug what the actual problem was yet.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

arm64

Shall we skip the test on non-amd64?

Tests bit-for-bit reproducibility of OCI tar

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
@AkihiroSuda AkihiroSuda force-pushed the testSourceDateEpochRepro branch from 1f76bc6 to 5786a98 Compare January 24, 2023 09:06
@AkihiroSuda
Copy link
Copy Markdown
Member Author

Updated to skip the test on non-amd64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants