upload-release: disable containerd image store to preserve gzip layer compression#15252
Conversation
… compression Docker 28+ defaults to the containerd image store, which pushes layers uncompressed instead of gzip. The GHA runner image updated Docker to 29.x (actions/runner-images#13633), causing the `nixos/nix:2.33.3` image to balloon from 138 MB to 505 MB, with all 70 layers pushed as `application/vnd.docker.image.rootfs.diff.tar` instead of `.tar.gzip`. OCI clients that only support gzip (e.g. `go-containerregistry`, used by Concourse CI) fail with "gzip: invalid header". This commit disables the containerd snapshotter in the release workflow before any Docker operations, restoring the classic storage driver that preserves gzip compression through the `docker load` / `docker push` pipeline. Fixes NixOS#15246
|
Thanks a lot! |
|
For future readers, this is the same fix as described in https://www.ytyng.com/en/blog/docker-29-ecr-push-403-forbidden-containerd-image-store/ |
xokdvium
added
backport 2.29-maintenance
|
Git push to origin failed for 2.29-maintenance with exitcode 1 |
|
Git push to origin failed for 2.30-maintenance with exitcode 1 |
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
xokdvium
added
backport 2.29-maintenance
|
Git push to origin failed for 2.29-maintenance with exitcode 1 |
1 similar comment
|
Git push to origin failed for 2.29-maintenance with exitcode 1 |
|
Git push to origin failed for 2.30-maintenance with exitcode 1 |
|
Git push to origin failed for 2.29-maintenance with exitcode 1 |
|
Git push to origin failed for 2.30-maintenance with exitcode 1 |
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
1 similar comment
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
|
Git push to origin failed for 2.30-maintenance with exitcode 1 |
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
2 similar comments
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
2 similar comments
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
|
Git push to origin failed for 2.29-maintenance with exitcode 1 |
|
Git push to origin failed for 2.30-maintenance with exitcode 1 |
|
Git push to origin failed for 2.31-maintenance with exitcode 1 |
|
Git push to origin failed for 2.32-maintenance with exitcode 1 |
|
Git push to origin failed for 2.33-maintenance with exitcode 1 |
…ssion upload-release: disable containerd image store to preserve gzip layer compression
3 participants
Motivation
Docker 28+ defaults to the containerd image store, which pushes layers uncompressed instead of gzip. The GHA runner image updated Docker to 29.x (actions/runner-images#13633), causing the
nixos/nix:2.33.3image to balloon from 138 MB to 505 MB, with all 70 layers pushed asapplication/vnd.docker.image.rootfs.diff.tarinstead of.tar.gzip. OCI clients that only support gzip (e.g.go-containerregistry, used by Concourse CI) fail with "gzip: invalid header".This commit disables the containerd snapshotter in the release workflow before any Docker operations, restoring the classic storage driver that preserves gzip compression through the
docker load/docker pushpipeline.Context
The Docker 29.x update to the containerd image store appears to have changed how
docker pushhandles layer compression, as there's a similar breaking change in push behavior being tracked in moby/moby#51532. I ran a before/after comparison on anubuntu-24.04runner with Docker 29.x, which confirms the fix, that is, pushing the same Hydra-builtnix:2.33.3tarball goes from 505 MB uncompressed (diff.tar) to 138 MB gzip (diff.tar.gzip).Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.