From 5f10fb29189f292c14c181290ac9c952c6d3ff81 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Thu, 19 May 2022 11:47:08 +0200 Subject: [PATCH 1/6] buildx: enhance wording around linux packages for workstations and servers Signed-off-by: CrazyMax --- buildx/working-with-buildx.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/buildx/working-with-buildx.md b/buildx/working-with-buildx.md index 51411faea63e..97d1ac290b91 100644 --- a/buildx/working-with-buildx.md +++ b/buildx/working-with-buildx.md @@ -18,7 +18,7 @@ multiple nodes concurrently. Docker Buildx is included in [Docker Desktop](../desktop/index.md) for Windows, macOS, and Linux. -### Static binaries +### Linux packages Docker Linux packages also include Docker Buildx when installed using the [DEB or RPM packages](../engine/install/index.md). @@ -31,8 +31,9 @@ Docker Linux packages also include Docker Buildx when installed using the [DEB o > installing buildx using manual download in production environments as they > will not be updated automatically with security updates. > -> On Windows, macOS, and Linux we recommend that you install [Docker Desktop](../desktop/index.md) -> instead. +> On Windows, macOS, and Linux workstations we recommend that you install +> [Docker Desktop](../desktop/index.md) instead. For Linux servers, we recommend +> that you follow the [instructions specific for your distribution](#linux-packages). {: .important} You can also download the latest binary from the [releases page on GitHub](https://github.com/docker/buildx/releases/latest){:target="_blank" rel="noopener" class="_"}. From e5c335c6222b838d9db264e25a650cd7cd5d5620 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Thu, 19 May 2022 12:58:13 +0200 Subject: [PATCH 2/6] build: create dedicated Docker Build section Signed-off-by: CrazyMax --- _data/toc.yaml | 11 +- .../buildx/index.md | 115 ++++-------------- build/buildx/install.md | 76 ++++++++++++ desktop/multi-arch.md | 2 +- desktop/previous-versions/2.x-mac.md | 2 +- desktop/previous-versions/2.x-windows.md | 2 +- go/buildx.md | 4 +- 7 files changed, 112 insertions(+), 100 deletions(-) rename buildx/working-with-buildx.md => build/buildx/index.md (66%) create mode 100644 build/buildx/install.md diff --git a/_data/toc.yaml b/_data/toc.yaml index 8a8246fd22db..3150399a2408 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1391,8 +1391,6 @@ manuals: - path: /engine/deprecated/ title: Deprecated features - - path: /buildx/working-with-buildx/ - title: Docker Buildx - path: /engine/context/working-with-contexts/ title: Docker Context - path: /engine/scan/ @@ -1400,6 +1398,15 @@ manuals: - path: /engine/sbom/ title: Docker SBOM (Experimental) +- sectiontitle: Docker Build + section: + - sectiontitle: Buildx component + section: + - path: /build/buildx/ + title: Working with Buildx + - path: /build/buildx/install/ + title: Install Buildx + - sectiontitle: Docker Compose section: - path: /compose/ diff --git a/buildx/working-with-buildx.md b/build/buildx/index.md similarity index 66% rename from buildx/working-with-buildx.md rename to build/buildx/index.md index 97d1ac290b91..498483db1205 100644 --- a/buildx/working-with-buildx.md +++ b/build/buildx/index.md @@ -1,7 +1,9 @@ --- -title: Docker Buildx +title: Buildx component description: Working with Docker Buildx -keywords: Docker, buildx, multi-arch +keywords: Docker, Buildx, BuildKit +redirect_from: +- /buildx/working-with-buildx/ --- ## Overview @@ -12,80 +14,7 @@ builder toolkit. It provides the same user experience as docker build with many new features like creating scoped builder instances and building against multiple nodes concurrently. -## Install - -### Docker Desktop - -Docker Buildx is included in [Docker Desktop](../desktop/index.md) for Windows, macOS, and Linux. - -### Linux packages - -Docker Linux packages also include Docker Buildx when installed using the [DEB or RPM packages](../engine/install/index.md). - -### Manual download - -> **Important** -> -> This section is for unattended installation of the buildx component. These -> instructions are mostly suitable for testing purposes. We do not recommend -> installing buildx using manual download in production environments as they -> will not be updated automatically with security updates. -> -> On Windows, macOS, and Linux workstations we recommend that you install -> [Docker Desktop](../desktop/index.md) instead. For Linux servers, we recommend -> that you follow the [instructions specific for your distribution](#linux-packages). -{: .important} - -You can also download the latest binary from the [releases page on GitHub](https://github.com/docker/buildx/releases/latest){:target="_blank" rel="noopener" class="_"}. - -Rename the relevant binary and copy it to the destination matching your OS: - -| OS | Binary name | Destination folder | -| -------- | -------------------- | -----------------------------------------| -| Linux | `docker-buildx` | `$HOME/.docker/cli-plugins` | -| macOS | `docker-buildx` | `$HOME/.docker/cli-plugins` | -| Windows | `docker-buildx.exe` | `%USERPROFILE%\.docker\cli-plugins` | - -Or copy it into one of these folders for installing it system-wide. - -On Unix environments: - -* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins` -* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins` - -On Windows: - -* `C:\ProgramData\Docker\cli-plugins` -* `C:\Program Files\Docker\cli-plugins` - -> **Note** -> -> On Unix environments, it may also be necessary to make it executable with `chmod +x`: -> ```shell -> $ chmod +x ~/.docker/cli-plugins/docker-buildx -> ``` - -### Dockerfile - -Here is how to install and use Buildx inside a Dockerfile through the -[`docker/buildx-bin`](https://hub.docker.com/r/docker/buildx-bin) image: - -```dockerfile -FROM docker -COPY --from=docker/buildx-bin:latest /buildx /usr/libexec/docker/cli-plugins/docker-buildx -RUN docker buildx version -``` - -## Set buildx as the default builder - -Running the command [`docker buildx install`](../engine/reference/commandline/buildx_install.md) -sets up docker builder command as an alias to `docker buildx`. This results in -the ability to have [`docker build`](../engine/reference/commandline/build.md) -use the current buildx builder. - -To remove this alias, run [`docker buildx uninstall`](../engine/reference/commandline/buildx_uninstall.md). - -## Build with buildx +## Build with Buildx To start a new build, run the command `docker buildx build .` @@ -129,23 +58,23 @@ the shared daemon, or for isolating builds for different projects. You can creat a new instance for a set of remote nodes, forming a build farm, and quickly switch between them. -You can create new instances using the [`docker buildx create`](../engine/reference/commandline/buildx_create.md) +You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md) command. This creates a new builder instance with a single node based on your current configuration. To use a remote node you can specify the `DOCKER_HOST` or the remote context name while creating the new builder. After creating a new instance, you can manage its -lifecycle using the [`docker buildx inspect`](../engine/reference/commandline/buildx_inspect.md), -[`docker buildx stop`](../engine/reference/commandline/buildx_stop.md), and -[`docker buildx rm`](../engine/reference/commandline/buildx_rm.md) commands. -To list all available builders, use [`docker buildx ls`](../engine/reference/commandline/buildx_ls.md). +lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md), +[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and +[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands. +To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md). After creating a new builder you can also append new nodes to it. -To switch between different builders, use [`docker buildx use `](../engine/reference/commandline/buildx_use.md). +To switch between different builders, use [`docker buildx use `](../../engine/reference/commandline/buildx_use.md). After running this command, the build commands will automatically use this builder. -Docker also features a [`docker context`](../engine/reference/commandline/context.md) +Docker also features a [`docker context`](../../engine/reference/commandline/context.md) command that you can use to provide names for remote Docker API endpoints. Buildx integrates with `docker context` to ensure all the contexts automatically get a default builder instance. You can also set the context name as the target when @@ -164,8 +93,8 @@ platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or When the current builder instance is backed by the `docker-container` driver, you can specify multiple platforms together. In this case, it builds a manifest list which contains images for all specified architectures. When you use this -image in [`docker run`](../engine/reference/commandline/run.md) or -[`docker service`](../engine/reference/commandline/service.md), Docker picks +image in [`docker run`](../../engine/reference/commandline/run.md) or +[`docker service`](../../engine/reference/commandline/service.md), Docker picks the correct image based on the node's platform. You can build multi-platform images using three different strategies that are @@ -183,7 +112,7 @@ it automatically loads it through a binary registered in the `binfmt_misc` handler. For QEMU binaries registered with `binfmt_misc` on the host OS to work -transparently inside containers, they must be statically compiled and registered with the `fix_binary` flag. +transparently inside containers, they must be statically compiled and registered with the `fix_binary` flag. This requires a kernel >= 4.8 and binfmt-support >= 2.1.7. You can check for proper registration by checking if `F` is among the flags in `/proc/sys/fs/binfmt_misc/qemu-*`. While Docker Desktop comes preconfigured @@ -208,7 +137,7 @@ $ docker buildx create --append --name mybuild node-arm64 $ docker buildx build --platform linux/amd64,linux/arm64 . ``` -Finally, depending on your project, the language that you use may have good +Finally, depending on your project, the language that you use may have good support for cross-compilation. In that case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with `--platform` using the native architecture of the build node. A list of build @@ -236,15 +165,15 @@ work. The build commands can be combined with general-purpose command runners (for example, `make`). However, these tools generally invoke builds in sequence and therefore cannot leverage the full potential of BuildKit parallelization, or combine BuildKit’s output for the user. For this use case, we have added a -command called [`docker buildx bake`](../engine/reference/commandline/buildx_bake.md). +command called [`docker buildx bake`](../../engine/reference/commandline/buildx_bake.md). -The `bake` command supports building images from compose files, similar to -[`docker-compose build`](../compose/reference/build.md), but allowing all the +The `bake` command supports building images from compose files, similar to +[`docker-compose build`](../../compose/reference/build.md), but allowing all the services to be built concurrently as part of a single request. There is also support for custom build rules from HCL/JSON files allowing better code reuse and different target groups. The design of bake is in very -early stages and we are looking for feedback from users. Let us know your -feedback by creating an issue in the -[Docker buildx](https://github.com/docker/buildx/issues){:target="_blank" rel="noopener" class="_"} +early stages and we are looking for feedback from users. Let us know your +feedback by creating an issue in the +[Docker buildx](https://github.com/docker/buildx/issues){:target="_blank" rel="noopener" class="_"} GitHub repository. diff --git a/build/buildx/install.md b/build/buildx/install.md new file mode 100644 index 000000000000..9a64c6c985f2 --- /dev/null +++ b/build/buildx/install.md @@ -0,0 +1,76 @@ +--- +title: Install Docker Buildx +description: How to install Docker Buildx +keywords: Docker, Buildx, BuildKit +--- + +## Docker Desktop + +Docker Buildx is included in [Docker Desktop](../../desktop/index.md) for Windows, macOS, and Linux. + +## Linux packages + +Docker Linux packages also include Docker Buildx when installed using the [DEB or RPM packages](../../engine/install/index.md). + +## Manual download + +> **Important** +> +> This section is for unattended installation of the buildx component. These +> instructions are mostly suitable for testing purposes. We do not recommend +> installing buildx using manual download in production environments as they +> will not be updated automatically with security updates. +> +> On Windows, macOS, and Linux workstations we recommend that you install +> [Docker Desktop](../../desktop/index.md) instead. For Linux servers, we recommend +> that you follow the [instructions specific for your distribution](#linux-packages). +{: .important} + +You can also download the latest binary from the [releases page on GitHub](https://github.com/docker/buildx/releases/latest){:target="_blank" rel="noopener" class="_"}. + +Rename the relevant binary and copy it to the destination matching your OS: + +| OS | Binary name | Destination folder | +| -------- | -------------------- | -----------------------------------------| +| Linux | `docker-buildx` | `$HOME/.docker/cli-plugins` | +| macOS | `docker-buildx` | `$HOME/.docker/cli-plugins` | +| Windows | `docker-buildx.exe` | `%USERPROFILE%\.docker\cli-plugins` | + +Or copy it into one of these folders for installing it system-wide. + +On Unix environments: + +* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins` +* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins` + +On Windows: + +* `C:\ProgramData\Docker\cli-plugins` +* `C:\Program Files\Docker\cli-plugins` + +> **Note** +> +> On Unix environments, it may also be necessary to make it executable with `chmod +x`: +> ```shell +> $ chmod +x ~/.docker/cli-plugins/docker-buildx +> ``` + +## Dockerfile + +Here is how to install and use Buildx inside a Dockerfile through the +[`docker/buildx-bin`](https://hub.docker.com/r/docker/buildx-bin) image: + +```dockerfile +FROM docker +COPY --from=docker/buildx-bin:latest /buildx /usr/libexec/docker/cli-plugins/docker-buildx +RUN docker buildx version +``` + +## Set buildx as the default builder + +Running the command [`docker buildx install`](../../engine/reference/commandline/buildx_install.md) +sets up docker builder command as an alias to `docker buildx`. This results in +the ability to have [`docker build`](../../engine/reference/commandline/build.md) +use the current buildx builder. + +To remove this alias, run [`docker buildx uninstall`](../../engine/reference/commandline/buildx_uninstall.md). diff --git a/desktop/multi-arch.md b/desktop/multi-arch.md index ed42c671d3e1..f5537a621743 100644 --- a/desktop/multi-arch.md +++ b/desktop/multi-arch.md @@ -46,7 +46,7 @@ than just native images. Buildx accomplishes this by adding new builder instances based on BuildKit, and leveraging Docker Desktop's technology stack to run non-native binaries. -For more information about the Buildx CLI command, see [Buildx](../buildx/working-with-buildx.md) +For more information about the Buildx CLI command, see [Buildx](../build/buildx/index.md). and the [`docker buildx` command line reference](../engine/reference/commandline/buildx.md). ### Build and run multi-architecture images diff --git a/desktop/previous-versions/2.x-mac.md b/desktop/previous-versions/2.x-mac.md index 79d1d7168cfe..be6b41432810 100644 --- a/desktop/previous-versions/2.x-mac.md +++ b/desktop/previous-versions/2.x-mac.md @@ -435,7 +435,7 @@ Note that you must sign in and create a Docker ID in order to download Docker De Docker Desktop Community 2.1.0.0 contains the following experimental features. * Docker App: Docker App is a CLI plugin that helps configure, share, and install applications. For more information, see [Working with Docker App](/app/working-with-app/). -* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Working with Docker Buildx](/buildx/working-with-buildx/). +* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Buildx component](/build/buildx/). ### Bug fixes and minor changes diff --git a/desktop/previous-versions/2.x-windows.md b/desktop/previous-versions/2.x-windows.md index bfbea3fc54fd..2cc25b565b27 100644 --- a/desktop/previous-versions/2.x-windows.md +++ b/desktop/previous-versions/2.x-windows.md @@ -557,7 +557,7 @@ Note that you must sign in and create a Docker ID in order to download Docker De Docker Desktop Community 2.1.0.0 contains the following experimental features: * Docker App: Docker App is a CLI plugin that helps configure, share, and install applications. For more information, see [Working with Docker App](/app/working-with-app/). -* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Working with Docker Buildx](/buildx/working-with-buildx/). +* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Buildx component](/build/buildx/). ### Bug fixes and minor changes diff --git a/go/buildx.md b/go/buildx.md index 537cffc3d7eb..1defe9d067ae 100644 --- a/go/buildx.md +++ b/go/buildx.md @@ -1,6 +1,6 @@ --- title: How to install Buildx description: Instructions on installing Buildx -keywords: Docker, buildx, multi-arch -redirect_to: /buildx/working-with-buildx/#install +keywords: Docker, Buildx, BuildKit +redirect_to: /build/buildx/install/ --- From 26aac8ffd6d60a53ddfc44e8410e86328d4e9078 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Thu, 19 May 2022 12:00:23 +0200 Subject: [PATCH 3/6] chore: add build team as code owners of build folder Signed-off-by: CrazyMax --- .github/CODEOWNERS | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a932346f6ad0..5d8a1d5e2228 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -3,6 +3,8 @@ # For more details, see https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners +/build @dockertopia @docker/build + /compose/ @dockertopia @usha-mandya /desktop/ @aevesdocker @usha-mandya From 9f11e13fa2e92982cd8e8a348ff07f05a303cc96 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Thu, 19 May 2022 12:59:05 +0200 Subject: [PATCH 4/6] build: fetch guides from upstream repo Signed-off-by: CrazyMax --- _config.yml | 8 ++++++++ _data/toc.yaml | 14 ++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/_config.yml b/_config.yml index a10b5bd3235b..62d046af9ea0 100644 --- a/_config.yml +++ b/_config.yml @@ -150,6 +150,14 @@ fetch-remote: - "!docs/README.md" # readme to make things nice in the compose-cli repo, but meaningless here - "!docs/architecture.md" # Compose-CLI architecture, unrelated to cloud integration + - repo: "https://github.com/docker/buildx" + ref: "master" + paths: + - dest: "build" + src: + - "docs/guides/*.md" + - "!docs/guides/cicd.md" # CI/CD documentation is already handled in ci-cd/github-actions + - repo: "https://github.com/docker/extensions-sdk" ref: "main" paths: diff --git a/_data/toc.yaml b/_data/toc.yaml index 3150399a2408..192ced4de524 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1406,6 +1406,20 @@ manuals: title: Working with Buildx - path: /build/buildx/install/ title: Install Buildx + - path: /build/cni-networking/ + title: CNI networking + - path: /build/custom-network/ + title: Using a custom network + - path: /build/custom-registry-config/ + title: Using a custom registry config + - path: /build/opentelemetry/ + title: OpenTelemetry support + - path: /build/registry-mirror/ + title: Define a registry mirror + - path: /build/remote-builder/ + title: Remote builder + - path: /build/resource-limiting/ + title: Resource limiting - sectiontitle: Docker Compose section: From 9142bc0d7cfbc13e99b128fb35366a11b98d6a59 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Wed, 1 Jun 2022 20:31:30 +0200 Subject: [PATCH 5/6] build: add bake guides Signed-off-by: CrazyMax --- _config.yml | 2 +- _data/toc.yaml | 14 ++++++++++++++ build/buildx/index.md | 25 +++---------------------- 3 files changed, 18 insertions(+), 23 deletions(-) diff --git a/_config.yml b/_config.yml index 62d046af9ea0..adf5a3b65113 100644 --- a/_config.yml +++ b/_config.yml @@ -155,7 +155,7 @@ fetch-remote: paths: - dest: "build" src: - - "docs/guides/*.md" + - "docs/guides/**" - "!docs/guides/cicd.md" # CI/CD documentation is already handled in ci-cd/github-actions - repo: "https://github.com/docker/extensions-sdk" diff --git a/_data/toc.yaml b/_data/toc.yaml index 192ced4de524..7d84fbc353f4 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1406,6 +1406,20 @@ manuals: title: Working with Buildx - path: /build/buildx/install/ title: Install Buildx + - sectiontitle: Bake + section: + - path: /build/bake/ + title: Overview + - path: /build/bake/file-definition/ + title: File definition + - path: /build/bake/configuring-build/ + title: Configuring builds + - path: /build/bake/hcl-vars-funcs/ + title: HCL variables and functions + - path: /build/bake/build-contexts/ + title: Build contexts and linking targets + - path: /build/bake/compose-xbake/ + title: Extension field with Compose - path: /build/cni-networking/ title: CNI networking - path: /build/custom-network/ diff --git a/build/buildx/index.md b/build/buildx/index.md index 498483db1205..7ed8b0d9054d 100644 --- a/build/buildx/index.md +++ b/build/buildx/index.md @@ -155,25 +155,6 @@ FROM alpine COPY --from=build /log /log ``` -## High-level build options - -Buildx also aims to provide support for high-level build concepts that go beyond -invoking a single build command. - -BuildKit efficiently handles multiple concurrent build requests and de-duplicating -work. The build commands can be combined with general-purpose command runners -(for example, `make`). However, these tools generally invoke builds in sequence -and therefore cannot leverage the full potential of BuildKit parallelization, -or combine BuildKit’s output for the user. For this use case, we have added a -command called [`docker buildx bake`](../../engine/reference/commandline/buildx_bake.md). - -The `bake` command supports building images from compose files, similar to -[`docker-compose build`](../../compose/reference/build.md), but allowing all the -services to be built concurrently as part of a single request. - -There is also support for custom build rules from HCL/JSON files allowing -better code reuse and different target groups. The design of bake is in very -early stages and we are looking for feedback from users. Let us know your -feedback by creating an issue in the -[Docker buildx](https://github.com/docker/buildx/issues){:target="_blank" rel="noopener" class="_"} -GitHub repository. +## High-level build options with Bake + +See [our guide](/build/bake/) to get started with Buildx Bake. From 1e61a78f690278e8b2f656d7bd69867376aa7066 Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Sat, 21 May 2022 20:31:17 +0200 Subject: [PATCH 6/6] use my fork to check bake docs Signed-off-by: CrazyMax --- _config.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_config.yml b/_config.yml index adf5a3b65113..31d5f6b56e95 100644 --- a/_config.yml +++ b/_config.yml @@ -150,8 +150,8 @@ fetch-remote: - "!docs/README.md" # readme to make things nice in the compose-cli repo, but meaningless here - "!docs/architecture.md" # Compose-CLI architecture, unrelated to cloud integration - - repo: "https://github.com/docker/buildx" - ref: "master" + - repo: "https://github.com/crazy-max/buildx" + ref: "bake-docs" paths: - dest: "build" src: