Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ jobs:
run: |
./hack/cross
env:
PLATFORMS: ${{ env.PLATFORMS }},darwin/amd64,windows/amd64
PLATFORMS: ${{ env.PLATFORMS }},darwin/amd64,darwin/arm64,windows/amd64,windows/arm64
RUNC_PLATFORMS: ${{ env.PLATFORMS }}
CACHEDIR_FROM: /tmp/.buildkit-cache/${{ env.CACHEKEY_CROSS }}
CACHEDIR_TO: /tmp/.buildkit-cache/${{ env.CACHEKEY_CROSS }}-new
Expand Down Expand Up @@ -382,7 +382,7 @@ jobs:
run: |
./hack/release-tar "${{ needs.release-base.outputs.tag }}" release-out
env:
PLATFORMS: ${{ env.PLATFORMS }},darwin/amd64,windows/amd64
PLATFORMS: ${{ env.PLATFORMS }},darwin/amd64,darwin/arm64,windows/amd64,windows/arm64
CACHEDIR_FROM: /tmp/.buildkit-cache/${{ env.CACHEKEY_BINARIES }} /tmp/.buildkit-cache/${{ env.CACHEKEY_CROSS }}
-
name: Move artifacts
Expand Down
122 changes: 56 additions & 66 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,48 +19,24 @@ ARG ALPINE_VERSION=3.12
FROM --platform=$BUILDPLATFORM alpine:${ALPINE_VERSION} AS git
RUN apk add --no-cache git

# xgo is a helper for golang cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:golang@sha256:6f7d999551dd471b58f70716754290495690efa8421e0a1fcf18eb11d0c0a537 AS xgo
# xx is a helper for cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:golang@sha256:810dc54d5144f133a218e88e319184bf8b9ce01d37d46ddb37573e90decd9eef AS xx

FROM --platform=$BUILDPLATFORM golang:1.13-alpine AS gostable
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Can be another PR, but we seriously need to move away from Go 1.13)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tiborvass has a spreadsheet with all the possible EINTR for syscalls handlers for go1.13+ that needs to be confirmed to not have possible unfixed cases. @tiborvass please share with @AkihiroSuda

But yes, unrelated to this PR.

FROM --platform=$BUILDPLATFORM golang:1.16-alpine AS golatest

FROM gostable AS go-linux
FROM golatest AS go-darwin
FROM golatest AS go-windows-amd64
FROM golatest AS go-windows-386
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support windows 386?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only left amd64&arm64 in the CI. These are just to cover all possible options. Once we don't need to switch between go versions this can be all removed.

FROM golatest AS go-windows-arm
FROM --platform=$BUILDPLATFORM tonistiigi/golang:497feff1-alpine AS go-windows-arm64
FROM go-windows-${TARGETARCH} AS go-windows

# gobuild is base stage for compiling go/cgo
FROM --platform=$BUILDPLATFORM golang:1.13-buster AS gobuild-minimal
COPY --from=xgo / /
RUN apt-get update && apt-get install --no-install-recommends -y libseccomp-dev file

# on amd64 you can also cross-compile to other platforms
FROM gobuild-minimal AS gobuild-cross-amd64
RUN dpkg --add-architecture s390x && \
dpkg --add-architecture ppc64el && \
apt-get update && \
apt-get --no-install-recommends install -y \
gcc-s390x-linux-gnu libc6-dev-s390x-cross libseccomp-dev:s390x \
crossbuild-essential-ppc64el libseccomp-dev:ppc64el \
--no-install-recommends

FROM gobuild-minimal AS gobuild-cross-amd64-arm
RUN echo "deb http://deb.debian.org/debian buster-backports main" >> /etc/apt/sources.list
RUN apt-get update && apt-get install --no-install-recommends -y libseccomp2=2.4.4-1~bpo10+1 libseccomp-dev=2.4.4-1~bpo10+1
RUN dpkg --add-architecture armel && \
dpkg --add-architecture armhf && \
dpkg --add-architecture arm64 && \
apt-get update && \
apt-get --no-install-recommends install -y \
crossbuild-essential-armel libseccomp2:armel=2.4.4-1~bpo10+1 libseccomp-dev:armel=2.4.4-1~bpo10+1 \
crossbuild-essential-armhf libseccomp2:armhf=2.4.4-1~bpo10+1 libseccomp-dev:armhf=2.4.4-1~bpo10+1 \
crossbuild-essential-arm64 libseccomp2:arm64=2.4.4-1~bpo10+1 libseccomp-dev:arm64=2.4.4-1~bpo10+1 \
--no-install-recommends

# define all valid target configurations for compilation
FROM gobuild-minimal AS gobuild-amd64-amd64
FROM gobuild-minimal AS gobuild-arm-arm
FROM gobuild-minimal AS gobuild-s390x-s390x
FROM gobuild-minimal AS gobuild-ppc64le-ppc64le
FROM gobuild-minimal AS gobuild-arm64-arm64
FROM gobuild-cross-amd64-arm AS gobuild-amd64-arm
FROM gobuild-cross-amd64 AS gobuild-amd64-s390x
FROM gobuild-cross-amd64 AS gobuild-amd64-ppc64le
FROM gobuild-cross-amd64-arm AS gobuild-amd64-arm64
FROM gobuild-$BUILDARCH-$TARGETARCH AS gobuild-base
FROM go-${TARGETOS} AS gobuild-base
RUN apk add --no-cache file bash clang lld pkgconfig git make
COPY --from=xx / /

# runc source
FROM git AS runc-src
Expand All @@ -73,9 +49,13 @@ RUN git clone https://github.com/opencontainers/runc.git runc \
FROM gobuild-base AS runc
WORKDIR $GOPATH/src/github.com/opencontainers/runc
ARG TARGETPLATFORM
# gcc is only installed for libgcc
# lld has issues building static binaries for ppc so prefer ld for it
RUN set -e; xx-apk add musl-dev gcc libseccomp-dev; \
[ "$(xx-info arch)" != "ppc64le" ] || XX_CC_PREFER_LINKER=ld xx-clang --setup-target-triple
RUN --mount=from=runc-src,src=/usr/src/runc,target=. --mount=target=/root/.cache,type=cache \
CGO_ENABLED=1 go build -mod=vendor -ldflags '-extldflags -static' -tags 'apparmor seccomp netgo cgo static_build osusergo' -o /usr/bin/runc ./ && \
file /usr/bin/runc | grep "statically linked"
CGO_ENABLED=1 xx-go build -mod=vendor -ldflags '-extldflags -static' -tags 'apparmor seccomp netgo cgo static_build osusergo' -o /usr/bin/runc ./ && \
xx-verify --static /usr/bin/runc

FROM gobuild-base AS buildkit-base
WORKDIR /src
Expand All @@ -96,18 +76,18 @@ ARG TARGETPLATFORM
RUN --mount=target=. --mount=target=/root/.cache,type=cache \
--mount=target=/go/pkg/mod,type=cache \
--mount=source=/tmp/.ldflags,target=/tmp/.ldflags,from=buildkit-version \
set -x; go build -ldflags "$(cat /tmp/.ldflags)" -o /usr/bin/buildctl ./cmd/buildctl && \
file /usr/bin/buildctl && file /usr/bin/buildctl | egrep "statically linked|Mach-O|Windows"
xx-go build -ldflags "$(cat /tmp/.ldflags)" -o /usr/bin/buildctl ./cmd/buildctl && \
xx-verify --static /usr/bin/buildctl

# build buildkitd binary
FROM buildkit-base AS buildkitd
ARG TARGETPLATFORM
ARG BUILDKITD_TAGS
ARG TARGETPLATFORM
RUN --mount=target=. --mount=target=/root/.cache,type=cache \
--mount=target=/go/pkg/mod,type=cache \
--mount=source=/tmp/.ldflags,target=/tmp/.ldflags,from=buildkit-version \
go build -ldflags "$(cat /tmp/.ldflags) -extldflags '-static'" -tags "osusergo netgo static_build seccomp ${BUILDKITD_TAGS}" -o /usr/bin/buildkitd ./cmd/buildkitd && \
file /usr/bin/buildkitd | egrep "statically linked|Windows"
CGO_ENABLED=0 xx-go build -ldflags "$(cat /tmp/.ldflags) -extldflags '-static'" -tags "osusergo netgo static_build seccomp ${BUILDKITD_TAGS}" -o /usr/bin/buildkitd ./cmd/buildkitd && \
xx-verify --static /usr/bin/buildkitd

FROM scratch AS binaries-linux-helper
COPY --from=runc /usr/bin/runc /buildkit-runc
Expand Down Expand Up @@ -153,8 +133,10 @@ WORKDIR /usr/src
RUN git clone https://github.com/containerd/containerd.git containerd

FROM gobuild-base AS containerd-base
RUN apt-get --no-install-recommends install -y btrfs-progs libbtrfs-dev
WORKDIR /go/src/github.com/containerd/containerd
ARG TARGETPLATFORM
ENV CGO_ENABLED=1 BUILDTAGS=no_btrfs
RUN xx-apk add musl-dev gcc && xx-go --wrap

FROM containerd-base AS containerd
ARG CONTAINERD_VERSION
Expand Down Expand Up @@ -186,27 +168,30 @@ WORKDIR /go/src/github.com/rootless-containers/rootlesskit
ARG TARGETPLATFORM
RUN --mount=target=/root/.cache,type=cache \
git checkout -q "$ROOTLESSKIT_VERSION" && \
CGO_ENABLED=0 go build -o /rootlesskit ./cmd/rootlesskit && \
file /rootlesskit | grep "statically linked"
CGO_ENABLED=0 xx-go build -o /rootlesskit ./cmd/rootlesskit && \
xx-verify --static /rootlesskit

FROM gobuild-base AS stargz-snapshotter
ARG STARGZ_SNAPSHOTTER_VERSION
RUN git clone https://github.com/containerd/stargz-snapshotter.git /go/src/github.com/containerd/stargz-snapshotter
WORKDIR /go/src/github.com/containerd/stargz-snapshotter
ARG TARGETPLATFORM
RUN --mount=target=/root/.cache,type=cache \
git checkout -q "$STARGZ_SNAPSHOTTER_VERSION" && \
xx-go --wrap && \
mkdir /out && CGO_ENABLED=0 PREFIX=/out/ make && \
file /out/containerd-stargz-grpc | grep "statically linked" && \
file /out/ctr-remote | grep "statically linked"
xx-verify --static /out/containerd-stargz-grpc && \
xx-verify --static /out/ctr-remote

FROM --platform=$BUILDPLATFORM alpine:${ALPINE_VERSION} AS fuse-overlayfs
RUN apk add --no-cache curl
COPY --from=xx / /
ARG FUSEOVERLAYFS_VERSION
ARG TARGETARCH
RUN echo $TARGETARCH | sed -e s/^amd64$/x86_64/ -e s/^arm64$/aarch64/ -e s/^arm$/armv7l/ > /uname_m && \
mkdir /out && \
curl -sSL -o /out/fuse-overlayfs https://github.com/containers/fuse-overlayfs/releases/download/${FUSEOVERLAYFS_VERSION}/fuse-overlayfs-$(cat /uname_m) && \
chmod +x /out/fuse-overlayfs
ARG TARGETPLATFORM
RUN mkdir /out && \
curl -sSL -o /out/fuse-overlayfs https://github.com/containers/fuse-overlayfs/releases/download/${FUSEOVERLAYFS_VERSION}/fuse-overlayfs-$(xx-info march) && \
chmod +x /out/fuse-overlayfs && \
xx-verify --static /out/fuse-overlayfs

# Copy together all binaries needed for oci worker mode
FROM buildkit-export AS buildkit-buildkitd.oci_only
Expand Down Expand Up @@ -251,15 +236,17 @@ RUN curl -Ls https://github.com/containernetworking/plugins/releases/download/$C

FROM buildkit-base AS integration-tests-base
ENV BUILDKIT_INTEGRATION_ROOTLESS_IDPAIR="1000:1000"
RUN apt-get --no-install-recommends install -y uidmap sudo vim iptables fuse \
RUN apk add --no-cache shadow shadow-uidmap sudo vim iptables fuse \
&& useradd --create-home --home-dir /home/user --uid 1000 -s /bin/sh user \
&& echo "XDG_RUNTIME_DIR=/run/user/1000; export XDG_RUNTIME_DIR" >> /home/user/.profile \
&& mkdir -m 0700 -p /run/user/1000 \
&& chown -R user /run/user/1000 /home/user \
&& update-alternatives --set iptables /usr/sbin/iptables-legacy
&& ln -s /sbin/iptables-legacy /usr/bin/iptables \
&& xx-go --wrap
# musl is needed to directly use the registry binary that is built on alpine
ENV BUILDKIT_INTEGRATION_CONTAINERD_EXTRA="containerd-1.3=/opt/containerd-alt/bin"
ENV BUILDKIT_INTEGRATION_SNAPSHOTTER=stargz
ENV CGO_ENABLED=0
COPY --from=stargz-snapshotter /out/* /usr/bin/
COPY --from=rootlesskit /rootlesskit /usr/bin/
COPY --from=containerd-alt /out/containerd* /opt/containerd-alt/bin/
Expand All @@ -280,14 +267,17 @@ VOLUME /var/lib/buildkit
# newuidmap & newgidmap binaries (shadow-uidmap 4.7-r1) shipped with alpine cannot be executed without CAP_SYS_ADMIN,
# because the binaries are built without libcap-dev.
# So we need to build the binaries with libcap enabled.
FROM alpine:${ALPINE_VERSION} AS idmap
RUN apk add --no-cache autoconf automake build-base byacc gettext gettext-dev gcc git libcap-dev libtool libxslt
RUN git clone https://github.com/shadow-maint/shadow.git /shadow
WORKDIR /shadow
FROM --platform=$BUILDPLATFORM alpine:${ALPINE_VERSION} AS idmap
RUN apk add --no-cache git autoconf automake clang lld gettext-dev libtool make byacc binutils
COPY --from=xx / /
ARG SHADOW_VERSION
RUN git checkout $SHADOW_VERSION
RUN ./autogen.sh --disable-nls --disable-man --without-audit --without-selinux --without-acl --without-attr --without-tcb --without-nscd \
&& make \
RUN git clone https://github.com/shadow-maint/shadow.git /shadow && cd /shadow && git checkout $SHADOW_VERSION
WORKDIR /shadow
ARG TARGETPLATFORM
RUN xx-apk add --no-cache musl-dev gcc libcap-dev
RUN CC=$(xx-clang --print-target-triple)-clang ./autogen.sh --disable-nls --disable-man --without-audit --without-selinux --without-acl --without-attr --without-tcb --without-nscd --host $(xx-clang --print-target-triple) \
&& make -j $(nproc) \
&& xx-verify src/newuidmap src/newuidmap \
&& cp src/newuidmap src/newgidmap /usr/bin

# Rootless mode.
Expand Down
14 changes: 7 additions & 7 deletions frontend/dockerfile/cmd/dockerfile-frontend/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# syntax = docker/dockerfile:1.1-experimental

FROM --platform=$BUILDPLATFORM tonistiigi/xx:golang@sha256:6f7d999551dd471b58f70716754290495690efa8421e0a1fcf18eb11d0c0a537 AS xgo
# xx is a helper for cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:golang@sha256:810dc54d5144f133a218e88e319184bf8b9ce01d37d46ddb37573e90decd9eef AS xx

FROM --platform=$BUILDPLATFORM golang:1.13-buster AS base
COPY --from=xgo / /
FROM --platform=$BUILDPLATFORM golang:1.13-alpine AS base
COPY --from=xx / /
WORKDIR /src
ENV GOFLAGS=-mod=vendor

Expand All @@ -16,15 +17,14 @@ RUN --mount=target=. \
echo -n "${VERSION}" | tee /tmp/.version;

FROM base AS build
RUN apt-get update && apt-get --no-install-recommends install -y file
RUN apk add --no-cache file
ARG BUILDTAGS=""
ARG TARGETPLATFORM
ENV TARGETPLATFORM=$TARGETPLATFORM
RUN --mount=target=. --mount=type=cache,target=/root/.cache \
--mount=target=/go/pkg/mod,type=cache \
--mount=source=/tmp/.ldflags,target=/tmp/.ldflags,from=version \
CGO_ENABLED=0 go build -o /dockerfile-frontend -ldflags "-d $(cat /tmp/.ldflags)" -tags "$BUILDTAGS netgo static_build osusergo" ./frontend/dockerfile/cmd/dockerfile-frontend && \
file /dockerfile-frontend | grep "statically linked"
CGO_ENABLED=0 xx-go build -o /dockerfile-frontend -ldflags "-d $(cat /tmp/.ldflags)" -tags "$BUILDTAGS netgo static_build osusergo" ./frontend/dockerfile/cmd/dockerfile-frontend && \
xx-verify --static /dockerfile-frontend

FROM scratch AS release
LABEL moby.buildkit.frontend.network.none="true"
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ require (
golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9
golang.org/x/net v0.0.0-20201224014010-6772e930b67b
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e
// genproto: the actual version is replaced in replace()
google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -770,6 +770,8 @@ golang.org/x/sys v0.0.0-20201202213521-69691e467435/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c h1:VwygUrnw9jn88c4u8GD3rZQbqrP/tgas88tPUbBxQrk=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4 h1:EZ2mChiOa8udjfp6rRmswTbtZN/QzUQp4ptM4rnjHvc=
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
Expand Down
1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_aix.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gc_arm64.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gc_s390x.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gc_x86.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gccgo_arm64.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gccgo_s390x.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_gccgo_x86.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_linux.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_linux_mips64x.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_linux_noinit.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_linux_ppc64x.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_mips64x.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_mipsx.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/golang.org/x/sys/cpu/cpu_other_arm.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions vendor/golang.org/x/sys/cpu/cpu_other_arm64.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading