BuildKit 使用介绍

Docker 利用 BuildKit 扩展镜像构建功能，如构建多种系统架构的镜像。

介绍

Docker 18.09 引入的 BuildKit，19.03+ 版本开始可以使用 docker buildx build 命令利用 BuildKit 构建镜像。

源码：https://github.com/moby/buildkit/
镜像：https://hub.docker.com/r/moby/buildkit
Docker Hub 支持 buildx 实现

启用 BuildKit 特性

docker v20.10 后默认开启，查看版本是否支持 BuildKit，如下 buildx 字样

$ docker buildx version
github.com/docker/buildx v0.8.2-docker 6224def4dd2c3d347eee19db595348c50d7cb491
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  ...
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  ...
Server:
 ...
 Server Version: 20.10.17
 ...

若不出现，可以采用如下方式启用该特性

$ cat > ~/.docker/config.json <<EOF
{
  "experimental": "enabled"
}
EOF

原理

本质上 buildx 通过调用 buildkit 的 api 管理构建任务
Docker 在 Linux 系统架构下是不支持 arm 架构镜像，通过运行一个新的 buildkit 容器让其支持该特性；Docker 桌面版则无需进行此项设置（Mac）
- docker 通过在本地运行 buildkit 容器，实现支持多架构镜像构建
- buildkit 也可以以服务的方式单独运行，参考
  - Expose BuildKit as a TCP service
buildkit 镜像可参考：https://github.com/docker-practice/buildx
构建时，通过配置 --platform 标志来指定构建输出的目标架构（例如 linux/amd64、linux/arm64 或 darwin/amd64）

安装 buildkit

参考

buildkitd 守护进程需要安装以下组件：

runc 或 crun
容器（如果你想使用容器工作）

启动 buildkitd 守护进程：您需要在主机上以 root 用户身份运行 buildkitd

$ sudo buildkitd

配置为系统服务，参考

cat <<EOF > /usr/lib/systemd/system/buildkitd.service
[Unit]
Description=buildkitd
After=network.target

[Service]
ExecStart=/usr/local/buildkit/bin/buildkitd

[Install]
WantedBy=multi-user.target
EOF

# 重新加载Unit file
systemctl daemon-reload
# 启动服务
systemctl start buildkitd
# 开机自启动
systemctl enable buildkitd

buildctl 命令使用

Dockerfile

FROM alpine
RUN echo "built with BuildKit!" >  file
CMD ["/bin/sh"]

构建

buildctl build \
--frontend=dockerfile.v0 \
--local context=. \
--local dockerfile=. \
--output type=docker,name=myimage,dest=out.tar

# 将out.tar导入到containerd中的某个命名空间下
ctr -n test image import out.tar

# 查看镜像
ctr -n test image ls

使用

help

docker buildx ...

$ docker buildx

Usage:  docker buildx [OPTIONS] COMMAND

Extended build capabilities with BuildKit

Options:
      --builder string   Override the configured builder instance

Management Commands:
  imagetools  Commands to work on images in registry

Commands:
  bake        Build from a file
  build       Start a build
  create      Create a new builder instance
  du          Disk usage
  inspect     Inspect current builder instance
  ls          List builder instances
  prune       Remove build cache
  rm          Remove a builder instance
  stop        Stop builder instance
  use         Set the current builder instance
  version     Show buildx version information

Run 'docker buildx COMMAND --help' for more information on a command.

新建 builder 实例

moby/buildkit 默认镜像，镜像说明参考

$ docker buildx create --name builder --driver docker-container

国内加速镜像

$ docker buildx create --use --name=builder-cn --driver docker-container --driver-opt image=dockerpracticesig/buildkit:master

# 腾讯云优化
$ docker buildx create --use --name=builder-cn --driver docker-container --driver-opt image=dockerpracticesig/buildkit:master-tencent

查看 builder 实例

$ docker buildx ls
NAME/NODE  DRIVER/ENDPOINT             STATUS   PLATFORMS
builder    docker-container
  builder0 unix:///var/run/docker.sock inactive
default *  docker
  default  default                     running  linux/amd64, linux/386

使用 builder 实例

$ docker buildx use builder

# * 存在的为当前使用的
$ docker buildx ls
NAME/NODE  DRIVER/ENDPOINT             STATUS   PLATFORMS
builder *  docker-container
  builder0 unix:///var/run/docker.sock inactive
default    docker
  default  default                     running  linux/amd64, linux/386

Demo Dockerfile

Dockerfile

$ mkdir ~/demo
$ cd ~/demo
$ cat > Dockerfile <<EOF
FROM --platform=$TARGETPLATFORM alpine
RUN uname -a > /os.txt
CMD cat /os.txt
EOF

说明：

$TARGETPLATFORM 是内置变量，由 --platform 参数来指定
alpine 镜像支持的架构包括：
- linux/amd64
- linux/arm/v6
- linux/arm/v7
- linux/arm64/v8
- linux/386
- linux/ppc64le
- linux/s390x

构建镜像

本示例构建好的镜像见 https://hub.docker.com/r/xiexianbin/multi-hello/tags

# 登录docker hub
$ docker login

# 构建并推送
$ docker buildx build --platform linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/386,linux/ppc64le,linux/s390x -t xiexianbin/multi-hello . --push

说明：

--push 可选，直接推送镜像到 docker hub
构建过程，默认拉去 pulling image moby/buildkit:buildx-stable-1 镜像，本地启动容器 creating container buildx_buildkit_builder0 即为 buildkit 的 builder 容器
构建日志如下

docker buildx build xxx ...

root@xiexianbin_cn:~/demo# docker buildx build --platform linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/386,linux/ppc64le,linux/s390x -t xiexianbin/multi-hello .
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 109.1s (24/24) FINISHED
 => [internal] booting buildkit                                                                                 90.3s
 => => pulling image moby/buildkit:buildx-stable-1                                                              89.2s
 => => creating container buildx_buildkit_builder0                                                               1.1s
 => [internal] load build definition from Dockerfile                                                             0.0s
 => => transferring dockerfile: 100B                                                                             0.0s
 => [linux/arm/v7 internal] load metadata for docker.io/library/alpine:latest                                    8.9s
 => [linux/amd64 internal] load metadata for docker.io/library/alpine:latest                                    10.4s
 => [linux/s390x internal] load metadata for docker.io/library/alpine:latest                                     8.9s
 => [linux/ppc64le internal] load metadata for docker.io/library/alpine:latest                                   9.7s
 => [linux/arm/v6 internal] load metadata for docker.io/library/alpine:latest                                    9.7s
 => [linux/arm64 internal] load metadata for docker.io/library/alpine:latest                                    10.4s
 => [linux/386 internal] load metadata for docker.io/library/alpine:latest                                       9.7s
 => [internal] load .dockerignore                                                                                0.8s
 => => transferring context: 2B                                                                                  0.0s
 => [linux/386 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733f  4.6s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:95dc695758361a4038a2d9026959d72e1f531114edb0341be7ce47d912ef069e 3.24MB / 3.24MB                   4.4s
 => => extracting sha256:95dc695758361a4038a2d9026959d72e1f531114edb0341be7ce47d912ef069e                        0.1s
 => [linux/amd64 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c73  6.8s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 3.40MB / 3.40MB                   3.3s
 => => extracting sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de                        0.1s
 => [linux/ppc64le 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c  6.0s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:55353ca330e9474ce7b858eca6842bb540ef4a70b2981c2ed47eefb9ef4253ad 3.35MB / 3.35MB                   1.4s
 => => extracting sha256:55353ca330e9474ce7b858eca6842bb540ef4a70b2981c2ed47eefb9ef4253ad                        0.1s
 => [linux/arm/v6 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c7  3.7s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:af09961d4a43b504efc76e38b50918977c28be73eeb8b926247783a00e8b9f2f 3.14MB / 3.14MB                   3.5s
 => => extracting sha256:af09961d4a43b504efc76e38b50918977c28be73eeb8b926247783a00e8b9f2f                        0.2s
 => [linux/arm/v7 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c7  7.9s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:f8dec92eec42224ef9e6ca9c6207ea6b9195dcf93d06bd5ceff0f814b62bf064 2.90MB / 2.90MB                   4.2s
 => => extracting sha256:f8dec92eec42224ef9e6ca9c6207ea6b9195dcf93d06bd5ceff0f814b62bf064                        0.1s
 => [linux/s390x 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c73  5.3s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  0.0s
 => => sha256:8bed2eae372fe236061920d89ae1ce89695a12df84989113bcc7ce4bd9774456 3.21MB / 3.21MB                   5.1s
 => => extracting sha256:8bed2eae372fe236061920d89ae1ce89695a12df84989113bcc7ce4bd9774456                        0.1s
 => [linux/arm64 1/2] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c73  3.5s
 => => resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a  8.2s
 => => sha256:9fda8d8052c61740409c4bea888859c141fd8cc3f58ac61943144ff6d1681b2d 3.33MB / 3.33MB                   3.2s
 => => extracting sha256:9fda8d8052c61740409c4bea888859c141fd8cc3f58ac61943144ff6d1681b2d                        0.2s
 => [linux/arm64 2/2] RUN uname -a > /os.txt                                                                     0.2s
 => [linux/arm/v6 2/2] RUN uname -a > /os.txt                                                                    0.2s
 => [linux/386 2/2] RUN uname -a > /os.txt                                                                       0.1s
 => [linux/s390x 2/2] RUN uname -a > /os.txt                                                                     0.2s
 => [linux/ppc64le 2/2] RUN uname -a > /os.txt                                                                   0.2s
 => [linux/amd64 2/2] RUN uname -a > /os.txt                                                                     0.1s
 => [linux/arm/v7 2/2] RUN uname -a > /os.txt                                                                    0.2s

查看构建的镜像

需要指定 --pull 参数才能查看

$ docker buildx imagetools inspect xiexianbin/multi-hello
Name:      docker.io/xiexianbin/multi-hello:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:2b15dfe786cdecb59dcc1f710a776a0a3454c1c6923a02a11d0fae495efb7a82

Manifests:
  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:eeeafa444f5b56d0650b48f0368ebf9034ce1d923d7579c85117367aaf8c8c5d
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:a2dd3f3b7778cb1da2e1bc0560300f9dbd7c2dafcfe7186fa6748b6b3c7aa830
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v6

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:cae98fd2fc1b281dea54b1f353797004907a53b331060661ba8548cec4d1e642
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:754f3e4b7cdfb2ec2c36ce911330b5ea8ebc8f9166f96592ea077a4aa8864ca1
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:88671d1de992488e818a2fd224cf2d83a6f72c7b014494a17d75897f2d603ad6
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/386

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:6c3d94d4dcb3a61c105581f89ef31689708bf0b1e12eff7b943fa8dfb3baa592
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le

  Name:      docker.io/xiexianbin/multi-hello:latest@sha256:44a705da5e4e212c13b3b03fc651e463a2bcb92f1bc3422eecee3fe6599790c1
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

builder 状态

$ docker buildx ls
NAME/NODE  DRIVER/ENDPOINT             STATUS  PLATFORMS
builder *  docker-container
  builder0 unix:///var/run/docker.sock running linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default    docker
  default  default                     running linux/amd64, linux/386

Dockerfile

变量

Dockerfile 支持如下架构相关的变量

TARGETPLATFORM 构建镜像的目标平台，例如 linux/amd64, linux/arm/v7 等
- TARGETOS TARGETPLATFORM 的 OS 类型，例如 linux, windows
- TARGETARCH TARGETPLATFORM 的架构类型，例如 amd64, arm
- TARGETVARIANT TARGETPLATFORM 的变种，该变量可能为空，例如 v7
BUILDPLATFORM 构建镜像主机平台，例如 linux/amd64
- BUILDOS BUILDPLATFORM 的 OS 类型，例如 linux
- BUILDARCH BUILDPLATFORM 的架构类型，例如 amd64
- BUILDVARIANT BUILDPLATFORM 的变种，该变量可能为空，例如 v7

RUN –mount=type=cache

# syntax = docker/dockerfile:experimental
FROM node:alpine as builder
​
WORKDIR /app
​
COPY package.json /app/
​
RUN --mount=type=cache,target=/app/node_modules,id=my_app_npm_module,sharing=locked \
    --mount=type=cache,target=/root/.npm,id=npm_cache \
    npm i
​
COPY src /app/src
​
RUN --mount=type=cache,target=/app/node_modules,id=my_app_npm_module,sharing=locked \
  npm run build
​
FROM nginx:alpine
​
RUN --mount=type=cache,target=/root/.npm,from=builder,source=/app/dist \
  mkdir -p /app/dist && cp -r /root/.npm/* /app/dist

--mount=type=cache: 这是 BuildKit 特有的挂载语法。它告诉 Docker 引擎：请给我准备一个持久化的缓存卷。
- 这个缓存卷的内容在多次构建（Build）之间是保留的。
- 即使这一层（Layer）的缓存失效了（例如你修改了 package.json），这个挂载卷里的内容依然存在。
target=/root/.npm: 这是指定缓存挂载到容器内部的目录路径。
- 为什么是 /root/.npm？ 因为这是 npm 默认存放下载缓存（Local Cache）的地方（假设你是以 root 用户运行构建）。
- 注意：这里挂载的不是 node_modules，而是 npm 的下载缓存。npm install 会先去这个目录找有没有下载过的包，如果有就直接解压，不需要从互联网下载。

github action

参考

github-action-buildkit ...

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    steps:
      ...
      # 配置 QEMU 和 buildx 用于多架构镜像的构建
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Log in to the Container registry
        uses: docker/login-action@v1
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      # 根据输入自动生成 tag 和 label 等数据，说明见下
      - name: Extract metadata for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE }}

      # 构建并上传
      - name: Build and push
        uses: docker/build-push-action@v2
        with:
          context: .
          file: ./Dockerfile
          target: production
          builder: ${{ steps.buildx.outputs.name }}
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache

      - name: Inspect image
        run: |
          docker buildx imagetools inspect \
          ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ steps.meta.outputs.version }}

扩展

argo-workflows 使用 buildkit 参考
Earthly (Makefile + Dockerfile = Earthfile)

介绍