Error building python 3.6 slim

[PhasecoreX]

I have this Dockerfile:

FROM python:3.6-slim

RUN apt-get update && apt-get install -y --no-install-recommends make

When I run this command:

docker buildx build . --pull --platform linux/arm/v5

I get this output:

WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load                                                                                       
[+] Building 27.3s (6/6) FINISHED                                     
 => [internal] load build definition from Dockerfile                                                                                                                                                          0.0s                                                                        
 => => transferring dockerfile: 129B                                                                                                                                                                          0.0s                                                                        
 => [internal] load .dockerignore                                                                                                                                                                             0.0s                                                                        
 => => transferring context: 2B                                                                                                                                                                               0.0s                                                                        
 => [internal] load metadata for docker.io/library/python:3.6-slim                                                                                                                                            0.6s                                                                        
 => [auth] library/python:pull token for registry-1.docker.io                                                                                                                                                 0.0s                                                                        
 => CACHED [1/2] FROM docker.io/library/python:3.6-slim@sha256:200bc4213e83751cc4a5496e72573ce8173713fc73266a1fd3ec62bd378a5890                                                                               0.0s                                                                        
 => => resolve docker.io/library/python:3.6-slim@sha256:200bc4213e83751cc4a5496e72573ce8173713fc73266a1fd3ec62bd378a5890                                                                                      0.0s                                                                        
 => ERROR [2/2] RUN apt-get update && apt-get install -y --no-install-recommends make                                                                                                                        26.6s                                                                        
------                                                                
 > [2/2] RUN apt-get update && apt-get install -y --no-install-recommends make:                                                              
#6 0.582 Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]                                                 
#6 0.583 Get:2 http://deb.debian.org/debian buster InRelease [121 kB] 
#6 0.679 Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]                                                               
#6 2.670 Get:4 http://security.debian.org/debian-security buster/updates/main armel Packages [232 kB]                                        
#6 4.570 Get:5 http://deb.debian.org/debian buster/main armel Packages [7627 kB]                                                             
#6 6.556 Get:6 http://deb.debian.org/debian buster-updates/main armel Packages [7860 B]                                                      
#6 8.912 Fetched 8106 kB in 8s (962 kB/s)                             
#6 8.912 Reading package lists...                                     
#6 16.39 Reading package lists...                                     
#6 23.71 Building dependency tree...                                  
#6 24.28 Reading state information...                                 
#6 24.65 Suggested packages:                                          
#6 24.65   make-doc                                                   
#6 25.19 The following NEW packages will be installed:                                                                                       
#6 25.19   make                                                       
#6 25.39 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.                                                                      
#6 25.39 Need to get 327 kB of archives.                              
#6 25.39 After this operation, 1300 kB of additional disk space will be used.                                                                
#6 25.39 Get:1 http://deb.debian.org/debian buster/main armel make armel 4.2.1-1.2 [327 kB]                                                  
#6 26.17 debconf: delaying package configuration, since apt-utils is not installed                                                           
#6 26.29 Fetched 327 kB in 0s (2000 kB/s)                             
#6 26.37 Error while loading /usr/sbin/dpkg-split: No such file or directory                                                                 
#6 26.38 Error while loading /usr/sbin/dpkg-deb: No such file or directory                                                                   
#6 26.38 dpkg: error processing archive /var/cache/apt/archives/make_4.2.1-1.2_armel.deb (--unpack):                                         
#6 26.38  dpkg-deb --control subprocess returned error exit status 1                                                                         
#6 26.40 Errors were encountered while processing:                    
#6 26.40  /var/cache/apt/archives/make_4.2.1-1.2_armel.deb                                                                                   
#6 26.50 E: Sub-process /usr/bin/dpkg returned an error code (1)                                                                             
------                                                                
Dockerfile:3                                                          
--------------------                                                  
   1 |     FROM python:3.6-slim                                       
   2 |                                                                
   3 | >>> RUN apt-get update && apt-get install -y --no-install-recommends make                                                             
   4 |                                                                
--------------------                                                  
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c apt-get update && apt-get install -y --no-install-recommends make]: exit code: 100

Some observations:

It seems like it's looking for /usr/sbin/dpkg-split and /usr/sbin/dpkg-deb, even though those are located in the /usr/bin folder.

Platforms linux/arm/v7 and linux/arm/v8 both fail, however linux/amd64 does not fail.

Changing my Dockerfile to use FROM python:3.6 (instead of slim) builds correctly for all cases (armv5, armv7, arm64v8, amd64), and manually starting a container (after doing the multiarch qemu setup) with docker run -it --rm arm32v5/python:3.6-slim bash allows me to install make just fine.

I wasn't sure if this was a buildx issue or just a python-slim issue, but I was also able to do this as my Dockerfile:

FROM arm32v5/python:3.6-slim

RUN apt-get update && apt-get install -y --no-install-recommends make

and do a regular docker build:

docker build . --pull

and it builds correctly. So my guess is it has something to do with buildx? Unless arm32v5/python:3.6-slim is different than python:3.6-slim that buildx would be pulling.

This might be related to #493, not entirely sure.

[fidesachates]

I just ran into this as well, but I had builds running just fine 3 months ago. Today I updated my code and ran my build pipeline and this error happened; nothing about my build files changed. Unfortunately in the last 3 months, I probably have ran a number of updates on my build server. I know that's not helpful sadly.

In my case, it's python:3.7-slim and changing to 3.7 doesn't work because I get pip issues in my Dockerfile (I think unrelated)

[JesseBowling]

I suspect a PATH issue, but only appears to affect ARM builds?

Staring on December 15th, my CI builds started failing to build ARM-based images. Using the following Dockerfile:

FROM ubuntu:18.04

RUN echo ${PATH}
RUN apt-get update
RUN apt-get upgrade -y

Then building with:
docker buildx build --platform linux/arm/v8 --progress=plain .

results in an error:

#4 [1/4] FROM docker.io/library/ubuntu:18.04@sha256:fd25e706f3dea2a5ff705db...
#4 resolve docker.io/library/ubuntu:18.04@sha256:fd25e706f3dea2a5ff705dbc3353cf37f08307798f3e360a13e9385840f73fb3 done
#4 CACHED
#5 [2/4] RUN echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/...
#5 0.104 /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
<snip>
#7 29.17 debconf: delaying package configuration, since apt-utils is not installed
#7 29.30 Fetched 4565 kB in 18s (255 kB/s)
#7 29.38 Error while loading /usr/local/sbin/dpkg-split: No such file or directory
#7 29.38 Error while loading /usr/local/sbin/dpkg-deb: No such file or directory
#7 29.39 dpkg: error processing archive /var/cache/apt/archives/libc6_2.27-3ubuntu1.4_armhf.deb (--unpack):
#7 29.39  dpkg-deb --control subprocess returned error exit status 1
#7 29.40 Errors were encountered while processing:
#7 29.41  /var/cache/apt/archives/libc6_2.27-3ubuntu1.4_armhf.deb
#7 29.51 E: Sub-process /usr/bin/dpkg returned an error code (1)
#7 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c apt-get upgrade -y]: exit code: 100

You'll note that /usr/local/sbin is first in the path. If I adjust the PATH to have /usr/bin first in the PATH, the error will "move" and then fail to find tar.

FROM ubuntu:18.04
ENV PATH="/usr/bin:${PATH}"
RUN echo ${PATH}
RUN apt-get update
RUN apt-get upgrade -y

Results in:

#4 [1/4] FROM docker.io/library/ubuntu:18.04@sha256:fd25e706f3dea2a5ff705db...
#4 resolve docker.io/library/ubuntu:18.04@sha256:fd25e706f3dea2a5ff705dbc3353cf37f08307798f3e360a13e9385840f73fb3 done
#4 CACHED

#5 [2/4] RUN echo /usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bi...
#5 0.102 /usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#5 DONE 0.1s
<snip>
#7 27.33 Fetched 4565 kB in 16s (287 kB/s)
#7 27.50 Error while loading /usr/bin/tar: No such file or directory
#7 27.51 dpkg-deb: error: tar subprocess returned error exit status 1
#7 27.51 dpkg: error processing archive /var/cache/apt/archives/libc6_2.27-3ubuntu1.4_armhf.deb (--unpack):
#7 27.51  dpkg-deb --control subprocess returned error exit status 2
#7 27.53 Errors were encountered while processing:
#7 27.53  /var/cache/apt/archives/libc6_2.27-3ubuntu1.4_armhf.deb
#7 27.63 E: Sub-process /usr/bin/dpkg returned an error code (1)
#7 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c apt-get upgrade -y]: exit code: 100

[fidesachates]

@JesseBowling

In my experimentations, you run into issues if you create a builder. Things work fine if you use the default builder and don't multi platform build (not that you can with the default builder anyway). Is this your experience as well?

[JesseBowling]

@fidesachates It seems to be specific to armhf (arm/v7 and arm/v8) builds. ARM64 and AMD64 build fine.

I'm not sure if I understand your question enough to give you a more solid answer than that. I can say that I've been primarily using this via Github Actions (i.e., https://github.com/CommunityHoneyNetwork/rdphoney/actions/runs/418797571) and it first broke 22 days ago.

The snippets I provided in the original issue came from trying to recreate the issue as simply as possible on my Macbook, which may well have had a non-default builder.

[fidesachates]

@JesseBowling I concur arm64 builds work for me as well.

I'm running builds on a ubuntu server. On that server, your examples have no problems for me with or without the PATH edit. Only when I create a docker builder in order to build for more than one platform do I run into these issues (even if I use the created builder to only build for one platform).

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx create --name cibuilder --driver docker-container --use
docker buildx ls
docker buildx inspect --bootstrap

I tried pinning qemu docker image to 5.0.0-5 which was released 4 months ago, but that made no difference.

[JesseBowling]

Well. Not sure what this implies, but in my CI we were specifying platform targets like so:

DOCKER_PLATFORMS=linux/amd64,linux/arm/v7,linux/arm/v8,linux/arm64

And it was failing with the above. That was changed to this:

DOCKER_PLATFORMS=linux/amd64,linux/armhf,linux/arm64

and all appears to be working as intended. Not sure what that means, or why things changed 22 days ago, but things are working for me now.

I do wonder what changed to cause this behavior?

[logopk]

Well. Not sure what this implies, but in my CI we were specifying platform targets like so:

DOCKER_PLATFORMS=linux/amd64,linux/arm/v7,linux/arm/v8,linux/arm64

And it was failing with the above. That was changed to this:

DOCKER_PLATFORMS=linux/amd64,linux/armhf,linux/arm64

and all appears to be working as intended. Not sure what that means, or why things changed 22 days ago, but things are working for me now.

I do wonder what changed to cause this behavior?

Does not help building linux/arm/v7 on debian or boot2docker.

Interesting that it will work on Docker Desktop 3.0.3 on MacOS BigSur. That uses buildx v0.4.2-docker.

Tried this version but same error.

[logopk]

The last try lead me to view the buildkitd versions. while both images showed "moby/buildkit:buildx-stable-1", the BigSur-version was loaded 3 months ago and is on version 0.7.2.
The latest on my dev machine is 0.8.1.

[logopk]

While trying I saw that there is an update to docker 20.10.2.
Don't know if this did the trick, but now it works.

However in the process I also did a
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

[nicobo]

Hi everyone

Got my docker updated to latest (20.10.2, Ubuntu) , did the multiarch reset thing, but still get the error...
I've deleted and recreated my builder, rebooted... still the same.

I've tried playing with the PATH, moving binaries, changing base images, then given up...

Any news on this ?

[JesseBowling]

@nicobo Can you share what architectures you're trying to build for? For me, the issue went away when I switched linux\arm\v8 and linux\arm\v7 to linux\armhf ....