No option to skip the build cache

[moikot]

I have the following use-case:

1. I want to do dotnet restore for my project files, and cache the resulting image in a registry.
2. In the next steps, I want to use the cached layers from the first step for running integration tests and unit tests in parallel.

I don't change my dependencies often and I would like that the first step just to check if the cache is valid and only if it doesn't then proceed with the build. Yet, when I build using this command:

docker buildx build \
  --progress=plain --pull \
  --cache-to=type=registry,ref=foo/bar \
  --cache-from=type=registry,ref=foo/bar \
  --target test --secret id=nuget-config,src=/baz/nuget.config \
  .

I see that the layers are fetched from the registry cache, and then there is an attempt to update the cache but nothing pushed since nothing has changed. I wonder is it possible to check that nothing has changed without pulling the layers locally and speed the build up?

Dockerfile:

ARG BUILD_IMAGE=mcr.microsoft.com/dotnet/core/sdk:3.1-alpine
ARG BASE_IMAGE=mcr.microsoft.com/dotnet/core/runtime-deps:3.1-alpine

FROM $BUILD_IMAGE AS prep

COPY . /src
RUN mkdir ./proj && cd ./src && \
  find . -type f -a \( -iname "*.sln" -o -iname "*.csproj" -o -iname "nuget.config" \) \
    -exec cp --parents "{}" ../proj/ \;

FROM $BUILD_IMAGE AS build
WORKDIR /src

COPY --from=prep ./proj .
RUN --mount=type=secret,id=nuget-config,dst=/src/nuget.config dotnet restore
COPY --from=prep ./src .
...

Logs:

2020-05-14T10:55:00.2186824Z romantic_bhaskara
2020-05-14T10:55:00.2977143Z time="2020-05-14T10:55:00Z" level=warning msg="No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load"
2020-05-14T10:55:00.3212271Z #1 [internal] booting buildkit
2020-05-14T10:55:00.4717246Z #1 pulling image moby/buildkit:buildx-stable-1
2020-05-14T10:55:03.3299882Z #1 pulling image moby/buildkit:buildx-stable-1 2.9s done
2020-05-14T10:55:03.3300866Z #1 creating container buildx_buildkit_romantic_bhaskara0
2020-05-14T10:55:05.7130847Z #1 creating container buildx_buildkit_romantic_bhaskara0 2.4s done
2020-05-14T10:55:05.7131200Z #1 DONE 5.3s
2020-05-14T10:55:05.7131308Z 
2020-05-14T10:55:05.7134946Z #3 [internal] load .dockerignore
2020-05-14T10:55:05.7135172Z #3 transferring context: 92B done
2020-05-14T10:55:05.7135366Z #3 DONE 0.0s
2020-05-14T10:55:05.7135469Z 
2020-05-14T10:55:05.7135654Z #2 [internal] load build definition from Dockerfile
2020-05-14T10:55:05.7135939Z #2 transferring dockerfile: 2.40kB 0.0s done
2020-05-14T10:55:05.7136138Z #2 DONE 0.0s
2020-05-14T10:55:05.7136658Z 
2020-05-14T10:55:05.7138027Z #4 resolve image config for docker.io/docker/dockerfile:1.0-experimental
2020-05-14T10:55:08.2656367Z #4 DONE 2.5s
2020-05-14T10:55:08.2656619Z 
2020-05-14T10:55:08.2657547Z #5 docker-image://docker.io/docker/dockerfile:1.0-experimental@sha256:cbd64...
2020-05-14T10:55:08.2658246Z #5 resolve docker.io/docker/dockerfile:1.0-experimental@sha256:cbd6491240cc8894d25e366ba83da19df1187f975dc3a5c2f88ce888ca696174 done
2020-05-14T10:55:09.3166891Z #5 sha256:cbd6491240cc8894d25e366ba83da19df1187f975dc3a5c2f88ce888ca696174 2.03kB / 2.03kB done
2020-05-14T10:55:09.3167895Z #5 sha256:9f37bbf3e89214574a30356139381d312e2a8d80877a312d31a1446eb2bdfa4a 527B / 527B done
2020-05-14T10:55:09.3168320Z #5 sha256:45169296fe16c4be3e75f43e6276e8adc60ea9e0aad0910264f6e0681e9e55b2 0B / 7.65MB 0.1s
2020-05-14T10:55:09.3168931Z #5 sha256:70387ff0e5d2bb01c76837530c63b7a5e0c74fd054665da9e0611f176d456c21 811B / 811B done
2020-05-14T10:55:10.2182229Z #5 sha256:45169296fe16c4be3e75f43e6276e8adc60ea9e0aad0910264f6e0681e9e55b2 7.65MB / 7.65MB 1.0s done
2020-05-14T10:55:10.2183352Z #5 unpacking docker.io/docker/dockerfile:1.0-experimental@sha256:cbd6491240cc8894d25e366ba83da19df1187f975dc3a5c2f88ce888ca696174
2020-05-14T10:55:10.3676840Z #5 unpacking docker.io/docker/dockerfile:1.0-experimental@sha256:cbd6491240cc8894d25e366ba83da19df1187f975dc3a5c2f88ce888ca696174 0.2s done
2020-05-14T10:55:10.3677297Z #5 DONE 2.1s
2020-05-14T10:55:10.4786802Z 
2020-05-14T10:55:10.4787191Z #6 [internal] load .dockerignore
2020-05-14T10:55:10.4787407Z #6 DONE 0.0s
2020-05-14T10:55:10.4787510Z 
2020-05-14T10:55:10.4788187Z #7 [internal] load build definition from Dockerfile
2020-05-14T10:55:10.4788462Z #7 transferring dockerfile: 2.40kB done
2020-05-14T10:55:10.4788664Z #7 DONE 0.0s
2020-05-14T10:55:10.4788767Z 
2020-05-14T10:55:10.4789955Z #8 [internal] load metadata for mcr.microsoft.com/dotnet/core/sdk:3.1-alpin...
2020-05-14T10:55:10.6126813Z #8 DONE 0.1s
2020-05-14T10:55:10.9128817Z 
2020-05-14T10:55:10.9130196Z #11 [internal] load build context
2020-05-14T10:55:10.9132139Z #11 DONE 0.0s
2020-05-14T10:55:10.9132262Z 
2020-05-14T10:55:10.9133359Z #10 [build 1/4] FROM mcr.microsoft.com/dotnet/core/sdk:3.1-alpine@sha256:c12...
2020-05-14T10:55:10.9134942Z #10 resolve mcr.microsoft.com/dotnet/core/sdk:3.1-alpine@sha256:c12db5f56719047e26544e06775ee4dfcf94de6703f9a3ebe3979e3a4cbd0c2b done
2020-05-14T10:55:10.9135271Z #10 DONE 0.0s
2020-05-14T10:55:10.9135374Z 
2020-05-14T10:55:10.9135555Z #9 [internal] helper image for file operations
2020-05-14T10:55:10.9136261Z #9 resolve docker.io/docker/dockerfile-copy:v0.1.9@sha256:e8f159d3f00786604b93c675ee2783f8dc194bb565e61ca5788f6a6e9d304061
2020-05-14T10:55:11.8138961Z #9 resolve docker.io/docker/dockerfile-copy:v0.1.9@sha256:e8f159d3f00786604b93c675ee2783f8dc194bb565e61ca5788f6a6e9d304061 0.9s done
2020-05-14T10:55:11.8139393Z #9 DONE 0.9s
2020-05-14T10:55:11.8139604Z 
2020-05-14T10:55:11.8140441Z #17 importing cache manifest from foo/bar...
2020-05-14T10:55:14.3514985Z #17 DONE 3.5s
2020-05-14T10:55:14.5011342Z 
2020-05-14T10:55:14.5013782Z #11 [internal] load build context
2020-05-14T10:55:14.5014590Z #11 transferring context: 195.08kB 0.0s done
2020-05-14T10:55:14.5014926Z #11 DONE 0.0s
2020-05-14T10:55:14.5015199Z 
2020-05-14T10:55:14.5015475Z #12 [prep 2/3] COPY . /src
2020-05-14T10:55:14.5015775Z #12 CACHED
2020-05-14T10:55:14.5016008Z 
2020-05-14T10:55:14.5017469Z #13 [prep 3/3] RUN mkdir ./proj && cd ./src &&   find . -type f -a ( -iname ...
2020-05-14T10:55:14.5018031Z #13 CACHED
2020-05-14T10:55:14.5018488Z 
2020-05-14T10:55:14.5019058Z #14 [build 2/4] COPY --from=prep ./proj .
2020-05-14T10:55:14.5019398Z #14 CACHED
2020-05-14T10:55:14.5019623Z 
2020-05-14T10:55:14.5020168Z #15 [build 3/4] RUN --mount=type=secret,id=nuget-config,dst=/src/nuget.confi...
2020-05-14T10:55:14.5020670Z #15 CACHED
2020-05-14T10:55:14.5020816Z 
2020-05-14T10:55:14.5022232Z #16 [build 4/4] COPY --from=prep ./src .
2020-05-14T10:55:14.5022866Z #16 pulling sha256:e8bfd8bdcdc09b5c86e25f71b6977fcc5b4893d9f9cdf1628d828ac07e01bf3a
2020-05-14T10:55:14.5023250Z #16 pulling sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac
2020-05-14T10:55:14.5023597Z #16 pulling sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08
2020-05-14T10:55:14.5023958Z #16 pulling sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb
2020-05-14T10:55:14.5024297Z #16 pulling sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2
2020-05-14T10:55:14.5024676Z #16 pulling sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51
2020-05-14T10:55:14.5025255Z #16 pulling sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46
2020-05-14T10:55:14.5025607Z #16 pulling sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1
2020-05-14T10:55:14.5026131Z #16 pulling sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea
2020-05-14T10:55:14.5026472Z #16 pulling sha256:9b8140bc7f72eddf44de516e1aa5dce01cb8fd8f0c300e3c124e10674ced2556
2020-05-14T10:55:15.3861079Z #16 pulling sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea 1.0s done
2020-05-14T10:55:15.8360117Z #16 pulling sha256:e8bfd8bdcdc09b5c86e25f71b6977fcc5b4893d9f9cdf1628d828ac07e01bf3a 1.4s done
2020-05-14T10:55:16.2780740Z #16 pulling sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb 1.9s done
2020-05-14T10:55:16.6792078Z #16 pulling sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08 2.3s done
2020-05-14T10:55:17.1296115Z #16 pulling sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51 2.6s done
2020-05-14T10:55:17.5801930Z #16 pulling sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46 3.1s done
2020-05-14T10:55:17.6919463Z #16 pulling sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac 3.3s done
2020-05-14T10:55:18.8928206Z #16 pulling sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2 4.4s done
2020-05-14T10:55:29.4013602Z #16 pulling sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1 14.9s done
2020-05-14T10:55:54.5860513Z #16 pulling sha256:9b8140bc7f72eddf44de516e1aa5dce01cb8fd8f0c300e3c124e10674ced2556 39.4s done
2020-05-14T10:55:54.5861865Z #16 unpacking
2020-05-14T10:56:11.5369425Z #16 pulling sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac
2020-05-14T10:56:11.5370103Z #16 pulling sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08
2020-05-14T10:56:11.5402155Z #16 pulling sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb
2020-05-14T10:56:11.5402746Z #16 pulling sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2
2020-05-14T10:56:11.5403395Z #16 pulling sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51 done
2020-05-14T10:56:11.5406539Z #16 pulling sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46
2020-05-14T10:56:11.5407348Z #16 pulling sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1
2020-05-14T10:56:11.5408642Z #16 pulling sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea
2020-05-14T10:56:11.5409327Z #16 pulling sha256:9b8140bc7f72eddf44de516e1aa5dce01cb8fd8f0c300e3c124e10674ced2556 done
2020-05-14T10:56:11.5409680Z #16 unpacking 17.8s done
2020-05-14T10:56:11.6870569Z #16 pulling sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac done
2020-05-14T10:56:11.6871128Z #16 pulling sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08 done
2020-05-14T10:56:11.6871677Z #16 pulling sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb done
2020-05-14T10:56:11.6872532Z #16 pulling sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2 done
2020-05-14T10:56:11.6873099Z #16 pulling sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51 done
2020-05-14T10:56:11.6873470Z #16 pulling sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46 done
2020-05-14T10:56:11.6873836Z #16 pulling sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1 done
2020-05-14T10:56:11.6874218Z #16 pulling sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea done
2020-05-14T10:56:11.6874477Z #16 unpacking 0.0s done
2020-05-14T10:56:11.6874645Z #16 CACHED
2020-05-14T10:56:11.6874745Z 
2020-05-14T10:56:11.6874892Z #18 exporting cache
2020-05-14T10:56:11.6875081Z #18 preparing build cache for export done
2020-05-14T10:56:11.6875390Z #18 writing layer sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac
2020-05-14T10:56:13.0385938Z #18 writing layer sha256:231f75e2aa067be39a57dc3955db94d4a55986245f7937f0612bb0b9a3b660ac 1.3s done
2020-05-14T10:56:13.0386966Z #18 writing layer sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51
2020-05-14T10:56:13.2992425Z #18 writing layer sha256:473e95b53a41c3274f32b5bc3b30e42029307b0abec913541cf7d39b16de9a51 0.4s done
2020-05-14T10:56:13.2992822Z #18 writing layer sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2
2020-05-14T10:56:13.7396035Z #18 writing layer sha256:7358a702f918aefcd5313413254b55c95af2219247a87d97358e261a691f56c2 0.4s done
2020-05-14T10:56:13.7396688Z #18 writing layer sha256:9b8140bc7f72eddf44de516e1aa5dce01cb8fd8f0c300e3c124e10674ced2556
2020-05-14T10:56:14.1887365Z #18 writing layer sha256:9b8140bc7f72eddf44de516e1aa5dce01cb8fd8f0c300e3c124e10674ced2556 0.4s done
2020-05-14T10:56:14.1888373Z #18 writing layer sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46
2020-05-14T10:56:14.6389096Z #18 writing layer sha256:c910986b9cb646d519bf0cfbd5d83e89655055d40bb82b50a3449d14727aac46 0.4s done
2020-05-14T10:56:14.6389836Z #18 writing layer sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08
2020-05-14T10:56:15.0891238Z #18 writing layer sha256:cbdbe7a5bc2a134ca8ec91be58565ec07d037386d1f1d8385412d224deafca08 0.4s done
2020-05-14T10:56:15.0892134Z #18 writing layer sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb
2020-05-14T10:56:15.3893994Z #18 writing layer sha256:d853ac765862d58d765fba0d8488f73b8302863b6f7167ab81f699b9179da5cb 0.4s done
2020-05-14T10:56:15.3895423Z #18 writing layer sha256:e8bfd8bdcdc09b5c86e25f71b6977fcc5b4893d9f9cdf1628d828ac07e01bf3a
2020-05-14T10:56:15.8398056Z #18 writing layer sha256:e8bfd8bdcdc09b5c86e25f71b6977fcc5b4893d9f9cdf1628d828ac07e01bf3a 0.4s done
2020-05-14T10:56:15.8398598Z #18 writing layer sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1
2020-05-14T10:56:16.0996567Z #18 writing layer sha256:efc136409f4a92aa0ac472204a2c794a5d0511cf58fdce71a1efbaa5a092d2a1 0.4s done
2020-05-14T10:56:16.0997300Z #18 writing layer sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea
2020-05-14T10:56:16.5367217Z #18 writing layer sha256:f9d73f115946fd262f4f599d111dda1a3b8fef92273391f1727ac7946fbad7ea 0.4s done
2020-05-14T10:56:16.5367814Z #18 writing config sha256:ac93681b42197cc010d6805bfeeaa49be47c5ad811fbea56b2f00073e9394fed
2020-05-14T10:56:18.3183850Z #18 writing config sha256:ac93681b42197cc010d6805bfeeaa49be47c5ad811fbea56b2f00073e9394fed 1.8s done
2020-05-14T10:56:18.3184551Z #18 writing manifest sha256:07a92f932dadb09c3867e7f38f0eeea914662843c88796d98c9795af53e3b7b1
2020-05-14T10:56:19.8547303Z #18 writing manifest sha256:07a92f932dadb09c3867e7f38f0eeea914662843c88796d98c9795af53e3b7b1 1.5s done
2020-05-14T10:56:19.8547874Z #18 DONE 8.3s

[FernandoMiguel]

what do you mean by "check if the cache is valid" ?
you can remove --cache-from and it will build without cache.
you can also use --no-cache , I think

[moikot]

I meant the remote cache is corresponding to the current content of the Dockerfile and its context.

I don't need to pull anything to the local cache since I just need to validate the build cache in the registry and only if it's not valid, pull some layers, build and do the update.

At the begging of the build buildx says

time="2020-05-14T10:55:00Z" level=warning msg="No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load"

I don't need the build result locally at all not even in the build cache if the remote registry cache is valid.

[FernandoMiguel]

I think those messages are from a recent change, to make it clear.

buildkit wil pull from the registry any and only the layers it needs to locally rebuild the image.

there is no way, afaik, to test an image without it being rebuilt.

how would you expect it to test if it doesn't rebuild?
trying to understand your reasoning , to see if we are discussing the same thing

[moikot]

It's ok, I'll try to explain. This is my view on in:

1. Docker file is the same
2. The build context, all the files used for the previous build cached in the remote registry, is the same.

Therefore everything in the remote cache should be valid and should correspond to the result of the current build. And indeed, nothing is actually building since the build, after spending a lot of time on pulling the cached layers from the remote registry is saying:

2020-05-14T10:56:11.6874645Z #16 CACHED

Then it tries to push the layers back but again since nothing has changed, this step is fast (as expected) and the build completes.

I'm not building anything else on this VM after, nether I'm running any docker container based on this build, the VM it's just gets destroyed immediately.

I wonder why is it necessary to spend time on fetching layers from the remote cache if they are used only for checking the remote cache validity.

I'm not sure if it's possible to check without rebuilding but it seems like there is no actual build anyways since it stops at the message "CACHED".

[FernandoMiguel]

that is exactly the expected behaviour
but I'll let @tonistiigi say his thoughts on this.
maybe there is a margin for improvement.

but what you describe, is really what it should happen.

[tonistiigi]

I wonder why is it necessary to spend time on fetching layers from the remote cache if they are used only for checking the remote cache validity.

This has been implemented in moby/buildkit#1475