diff --git a/_data/toc.yaml b/_data/toc.yaml index f4f749bb282a..84b31399a478 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1395,10 +1395,13 @@ manuals: - sectiontitle: Buildx section: - path: /build/buildx/ - title: Working with Buildx + title: Buildx overview - path: /build/buildx/install/ title: Install Buildx - + - path: /build/buildx/multiple-builders/ + title: Using multiple builders + - path: /build/buildx/multiplatform-images/ + title: Building multi-platform images - sectiontitle: Docker Compose section: - path: /compose/ diff --git a/build/buildx/index.md b/build/buildx/index.md index 4dab0bb66329..e21f0e12e376 100644 --- a/build/buildx/index.md +++ b/build/buildx/index.md @@ -1,5 +1,5 @@ --- -title: Buildx +title: Working with Buildx description: Working with Docker Buildx keywords: build, buildx, buildkit redirect_from: @@ -47,115 +47,6 @@ automatically added to the "docker images" view by default, whereas when using other drivers, the method for outputting an image needs to be selected with `--output`. -## Work with builder instances - -By default, Buildx uses the `docker` driver if it is supported, providing a user -experience very similar to the native `docker build`. Note that you must use a -local shared daemon to build your applications. - -Buildx allows you to create new instances of isolated builders. You can use this -to get a scoped environment for your CI builds that does not change the state of -the shared daemon, or for isolating builds for different projects. You can create -a new instance for a set of remote nodes, forming a build farm, and quickly -switch between them. - -You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md) -command. This creates a new builder instance with a single node based on your -current configuration. - -To use a remote node you can specify the `DOCKER_HOST` or the remote context name -while creating the new builder. After creating a new instance, you can manage its -lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md), -[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and -[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands. -To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md). -After creating a new builder you can also append new nodes to it. - -To switch between different builders, use [`docker buildx use `](../../engine/reference/commandline/buildx_use.md). -After running this command, the build commands will automatically use this -builder. - -Docker also features a [`docker context`](../../engine/reference/commandline/context.md) -command that you can use to provide names for remote Docker API endpoints. Buildx -integrates with `docker context` to ensure all the contexts automatically get a -default builder instance. You can also set the context name as the target when -you create a new builder instance or when you add a node to it. - -## Build multi-platform images - -BuildKit is designed to work well for building for multiple platforms and not -only for the architecture and operating system that the user invoking the build -happens to run. - -When you invoke a build, you can set the `--platform` flag to specify the target -platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or -`darwin/amd64`). - -When the current builder instance is backed by the `docker-container` driver, -you can specify multiple platforms together. In this case, it builds a manifest -list which contains images for all specified architectures. When you use this -image in [`docker run`](../../engine/reference/commandline/run.md) or -[`docker service`](../../engine/reference/commandline/service.md), Docker picks -the correct image based on the node's platform. - -You can build multi-platform images using three different strategies that are -supported by Buildx and Dockerfiles: - -1. Using the QEMU emulation support in the kernel -2. Building on multiple native nodes using the same builder instance -3. Using a stage in Dockerfile to cross-compile to different architectures - -QEMU is the easiest way to get started if your node already supports it (for -example. if you are using Docker Desktop). It requires no changes to your -Dockerfile and BuildKit automatically detects the secondary architectures that -are available. When BuildKit needs to run a binary for a different architecture, -it automatically loads it through a binary registered in the `binfmt_misc` -handler. - -For QEMU binaries registered with `binfmt_misc` on the host OS to work -transparently inside containers, they must be statically compiled and registered -with the `fix_binary` flag. This requires a kernel >= 4.8 and -binfmt-support >= 2.1.7. You can check for proper registration by checking if -`F` is among the flags in `/proc/sys/fs/binfmt_misc/qemu-*`. While Docker -Desktop comes preconfigured with `binfmt_misc` support for additional platforms, -for other installations it likely needs to be installed using -[`tonistiigi/binfmt`](https://github.com/tonistiigi/binfmt){:target="_blank" rel="noopener" class="_"} -image. - -```console -$ docker run --privileged --rm tonistiigi/binfmt --install all -``` - -Using multiple native nodes provide better support for more complicated cases -that are not handled by QEMU and generally have better performance. You can -add additional nodes to the builder instance using the `--append` flag. - -Assuming contexts `node-amd64` and `node-arm64` exist in `docker context ls`; - -```console -$ docker buildx create --use --name mybuild node-amd64 -mybuild -$ docker buildx create --append --name mybuild node-arm64 -$ docker buildx build --platform linux/amd64,linux/arm64 . -``` - -Finally, depending on your project, the language that you use may have good -support for cross-compilation. In that case, multi-stage builds in Dockerfiles -can be effectively used to build binaries for the platform specified with -`--platform` using the native architecture of the build node. A list of build -arguments like `BUILDPLATFORM` and `TARGETPLATFORM` is available automatically -inside your Dockerfile and can be leveraged by the processes running as part -of your build. - -```dockerfile -# syntax=docker/dockerfile:1 -FROM --platform=$BUILDPLATFORM golang:alpine AS build -ARG TARGETPLATFORM -ARG BUILDPLATFORM -RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log -FROM alpine -COPY --from=build /log /log -``` ## High-level build options diff --git a/build/buildx/install.md b/build/buildx/install.md index 0f33bd40122b..b2fe45d0ffce 100644 --- a/build/buildx/install.md +++ b/build/buildx/install.md @@ -75,4 +75,4 @@ sets up docker builder command as an alias to `docker buildx`. This results in the ability to have [`docker build`](../../engine/reference/commandline/build.md) use the current buildx builder. -To remove this alias, run [`docker buildx uninstall`](../../engine/reference/commandline/buildx_uninstall.md). +To remove this alias, run [`docker buildx uninstall`](../../engine/reference/commandline/buildx_uninstall.md). \ No newline at end of file diff --git a/build/buildx/multiplatform-images.md b/build/buildx/multiplatform-images.md new file mode 100644 index 000000000000..9f0b3fc87e57 --- /dev/null +++ b/build/buildx/multiplatform-images.md @@ -0,0 +1,78 @@ +--- +title: Building multi-platform images +description: Different strategies for building multi-platform images +keywords: build, buildx, buildkit, multi-platform images +--- +BuildKit is designed to work well for building for multiple platforms and not +only for the architecture and operating system that the user invoking the build +happens to run. + +When you invoke a build, you can set the `--platform` flag to specify the target +platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or +`darwin/amd64`). + +When the current builder instance is backed by the `docker-container` driver, +you can specify multiple platforms together. In this case, it builds a manifest +list which contains images for all specified architectures. When you use this +image in [`docker run`](../../engine/reference/commandline/run.md) or +[`docker service`](../../engine/reference/commandline/service.md), Docker picks +the correct image based on the node's platform. + +You can build multi-platform images using three different strategies that are +supported by Buildx and Dockerfiles: + +1. Using the QEMU emulation support in the kernel +2. Building on multiple native nodes using the same builder instance +3. Using a stage in Dockerfile to cross-compile to different architectures + +QEMU is the easiest way to get started if your node already supports it (for +example. if you are using Docker Desktop). It requires no changes to your +Dockerfile and BuildKit automatically detects the secondary architectures that +are available. When BuildKit needs to run a binary for a different architecture, +it automatically loads it through a binary registered in the `binfmt_misc` +handler. + +For QEMU binaries registered with `binfmt_misc` on the host OS to work +transparently inside containers, they must be statically compiled and registered +with the `fix_binary` flag. This requires a kernel >= 4.8 and +binfmt-support >= 2.1.7. You can check for proper registration by checking if +`F` is among the flags in `/proc/sys/fs/binfmt_misc/qemu-*`. While Docker +Desktop comes preconfigured with `binfmt_misc` support for additional platforms, +for other installations it likely needs to be installed using +[`tonistiigi/binfmt`](https://github.com/tonistiigi/binfmt){:target="_blank" rel="noopener" class="_"} +image. + +```console +$ docker run --privileged --rm tonistiigi/binfmt --install all +``` + +Using multiple native nodes provide better support for more complicated cases +that are not handled by QEMU and generally have better performance. You can +add additional nodes to the builder instance using the `--append` flag. + +Assuming contexts `node-amd64` and `node-arm64` exist in `docker context ls`; + +```console +$ docker buildx create --use --name mybuild node-amd64 +mybuild +$ docker buildx create --append --name mybuild node-arm64 +$ docker buildx build --platform linux/amd64,linux/arm64 . +``` + +Finally, depending on your project, the language that you use may have good +support for cross-compilation. In that case, multi-stage builds in Dockerfiles +can be effectively used to build binaries for the platform specified with +`--platform` using the native architecture of the build node. A list of build +arguments like `BUILDPLATFORM` and `TARGETPLATFORM` is available automatically +inside your Dockerfile and can be leveraged by the processes running as part +of your build. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM --platform=$BUILDPLATFORM golang:alpine AS build +ARG TARGETPLATFORM +ARG BUILDPLATFORM +RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log +FROM alpine +COPY --from=build /log /log +``` \ No newline at end of file diff --git a/build/buildx/multiple-builders.md b/build/buildx/multiple-builders.md new file mode 100644 index 000000000000..4a68cf1555f9 --- /dev/null +++ b/build/buildx/multiple-builders.md @@ -0,0 +1,36 @@ +--- +title: Using multiple builders +description: How to instantiate and work with multiple builders +keywords: build, buildx, buildkit, builders, build drivers +--- +By default, Buildx uses the `docker` driver if it is supported, providing a user +experience very similar to the native `docker build`. Note that you must use a +local shared daemon to build your applications. + +Buildx allows you to create new instances of isolated builders. You can use this +to get a scoped environment for your CI builds that does not change the state of +the shared daemon, or for isolating builds for different projects. You can create +a new instance for a set of remote nodes, forming a build farm, and quickly +switch between them. + +You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md) +command. This creates a new builder instance with a single node based on your +current configuration. + +To use a remote node you can specify the `DOCKER_HOST` or the remote context name +while creating the new builder. After creating a new instance, you can manage its +lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md), +[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and +[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands. +To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md). +After creating a new builder you can also append new nodes to it. + +To switch between different builders, use [`docker buildx use `](../../engine/reference/commandline/buildx_use.md). +After running this command, the build commands will automatically use this +builder. + +Docker also features a [`docker context`](../../engine/reference/commandline/context.md) +command that you can use to provide names for remote Docker API endpoints. Buildx +integrates with `docker context` to ensure all the contexts automatically get a +default builder instance. You can also set the context name as the target when +you create a new builder instance or when you add a node to it. \ No newline at end of file diff --git a/build/hellobuild.md b/build/hellobuild.md index 880495dd13af..7cdcb96f2fad 100644 --- a/build/hellobuild.md +++ b/build/hellobuild.md @@ -1,7 +1,7 @@ --- title: Hello Build description: Build Hello World -keywords: build, buildx, buildkit, getting started, Dockerfile, image layers, build instructions, build context +keywords: build, buildx, buildkit, getting started, dockerfile, image layers, build instructions, build context --- ## Hello Build! diff --git a/build/index.md b/build/index.md index a00a854eb25e..40fa7fd6cda0 100644 --- a/build/index.md +++ b/build/index.md @@ -43,7 +43,8 @@ To get started with Build, see the [Hello Build](hellobuild.md) page. Run Buildx with different configurations depending on the scenario you are working on, regardless of whether you are using your local machine or a remote compute cluster, all from the comfort of your local working environment. Here’s a quick overview of the drivers and the use cases they support: * `docker`– use the default built-in builder to get started quickly. - * `docker-container` – spin up a dedicated builder in a docker container to unlock more advanced features like advanced caching, and multi-arch images. + * `docker-container` – spin up a dedicated builder in a docker container to unlock more advanced features like advanced caching, and multi-platform images. + * `kubernetes`– connect to your Kubernetes cluster to unlock more features and to run your builds at scale. * `remote`– manually manage your own BuildKit builder and connect directly to it. @@ -52,7 +53,7 @@ Improve build performance by using a persistent shared build cache to avoid repe * **Creating build-once, run-anywhere with multi-architecture builds** Collaborate across platforms with one build artifact. -See [Build multi platform images](buildx/index.md/#build-multi-platform-images). +See [Build multi-platform images](buildx/multiplatform-images.md). ### Automating your builds