使用 kaniko 在 Kubernetes 中构建 Docker 镜像

发布时间: 更新时间: 总字数:898 阅读时间:2m 作者: 分享 复制网址

kaniko是Google开源的一款在 Kubernetes 用来构建容器镜像的工具,kaniko不依赖于Docker daemon进程,完全在用户空间根据Dockerfile构建镜像。

介绍

kaniko以容器镜像的方式运行,特点:

  • 不依赖 Docker daemon 进程,构建容器镜像的工具
  • 用户空间(非特权模式)构建镜像,不需要 root 权限
  • 可复现的容器镜像构建

使用场景:

  • kubernetes 集群上构建镜像
  • jenkins pipeline 构建镜像
  • 与 Tekton 结合实现在K8s中制作镜像,参考中的链接(Tekton 使用示例)

Docker 使用

docker run -ti --rm -v `pwd`:/workspace -v `pwd`/config.json:/kaniko/.docker/config.json:ro gcriokaniko/executor:latest-1317 --dockerfile=Dockerfile --destination=yourimagename

kubernetes 使用

kubectl create secret generic kaniko-secret --from-file=<path to kaniko-secret.json>
kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>
apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcriokaniko/executor:latest-1317  # gcr.io/kaniko-project/executor:latest
    args:
    - "--dockerfile=<path to Dockerfile within the build context>"
    - "--context=gs://<GCS bucket>/<path to .tar.gz>"
    - "--destination=<gcr.io/$PROJECT/$IMAGE:$TAG>"
    volumeMounts:
    - name: kaniko-secret
      mountPath: /secret
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /secret/kaniko-secret.json
  restartPolicy: Never
  volumes:
  - name: kaniko-secret
    secret:
      secretName: kaniko-secret

kaniko 镜像在 gcr.io 上,可以在 https://mirrors.kb.cx/?s=kaniko-project 查看对应同步的镜像。

Docker Cache 示例

准备

  • 模拟 Docker file
mkdir code
cat << code/Dockerfile > EOF
FROM alpine:latest

RUN apk add curl
RUN apk add wget
EOF
  • 帮助命令
docker run -it --rm gcriokaniko/executor:v1.9.1 --help

制作镜像示例

docker run \
  -v "$HOME"/.docker/config.json:/kaniko/.docker/config.json \
  -v $(pwd)/code:/workspace \
  gcriokaniko/executor:v1.9.1 \
  --verbosity debug \
  --dockerfile /workspace/Dockerfile \
  --destination "harbor.xiexianbin.cn/xiexianbin/test-cache:1" \
  --context dir:///workspace/ \
  --insecure-registry harbor.xiexianbin.cn \
  --skip-tls-verify \
  --skip-tls-verify-pull \
  --cache=true \
  --cache-run-layers \
  --cache-ttl 10m
#   --cache-ttl default is 336h0m0s
#   --cache-repo "harbor.xiexianbin.cn/xiexianbin/test-cache/cache"

说明:

  • --context 类似于 docker build 的 context,如 dir:///workspace/

Kaniko cache 说明

基于 harbor 仓库

  • Kaniko 会根据每个 Dockerfile 中的 RUN 命令生成一个镜像,并立即提交到 docker hub 中,下次在使用时,会检测 Docker hub 是否有该镜像,若存在,直接pull并使用。该过程类似于 docker build 的 cache 过程
  • 每一个 RUN 生成一个镜像(即 Docker 中的层)
  • build 日志如下:
# 检测是否存在 cache
# DEBU[0011] Optimize: cache key for command RUN apk add curl 5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3
# INFO[0011] Checking for cached layer harbor.xiexianbin.cn/xiexianbin/test-cache/cache:5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3...
# INFO[0012] No cached layer found for cmd RUN apk add curl
...
# INFO[0030] Pushing layer harbor.xiexianbin.cn/xiexianbin/test-cache/cache:5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3 to cache now
# INFO[0031] Pushing image to xiexianbin/test-cache/cache:5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3
# INFO[0033] Pushed xiexianbin/test-cache/cache@sha256:5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3
  • 下次使用
# INFO[0004] Checking for cached layer harbor.xiexianbin.cn/xiexianbin/test-cache/cache:5ee6afd6e969b6190a31473c5c6afbf2f164e1fb3af30e17ba7d00d2721107b3...
# INFO[0006] Using caching version of cmd: RUN apk add curl

基于 cache dir

该功能用来缓存基础镜像到指定的目录,参考

docker run gcr.io/kaniko-project/warmer:latest --cache-dir=/cache --image alpine:3.8
  • Before
[36mkaniko-issue |[0m [37mDEBU[0m[0000] Built stage name to index map: map[]
[36mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image manifest alpine:3.8
[36mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image alpine:3.8
[36mkaniko-issue |[0m [37mDEBU[0m[0000] No file found for cache key sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4 stat /cache/sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4: no such file or directory
[36mkaniko-issue |[0m [37mDEBU[0m[0000] Image alpine:3.8 not found in cache
[36mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image manifest alpine:3.8
[36mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image alpine:3.8
[36mkaniko-issue |[0m [36mINFO[0m[0001] Built cross stage deps: map[]
[36mkaniko-issue |[0m [36mINFO[0m[0001] Retrieving image manifest alpine:3.8
[36mkaniko-issue |[0m [36mINFO[0m[0001] Retrieving image alpine:3.8
  • After
[33mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image manifest alpine:3.8
[33mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image alpine:3.8
[36mkaniko-cache |[0m [36mINFO[0m[0000] Found sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4 in local cache
[36mkaniko-cache |[0m [36mINFO[0m[0000] Found manifest at /cache/sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4.json
[33mkaniko-issue |[0m [36mINFO[0m[0000] Found sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4 in local cache
[33mkaniko-issue |[0m [36mINFO[0m[0000] Found manifest at /cache/sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4.json
[33mkaniko-issue |[0m [36mINFO[0m[0000] Built cross stage deps: map[]
[33mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image manifest alpine:3.8
[33mkaniko-issue |[0m [36mINFO[0m[0000] Retrieving image alpine:3.8

其他

  • 基于 PVC,如 java 的 .m2 文件夹或 nodejs 的 node_modules 文件夹可以通过 PVC 缓存

F&Q

Debug

docker run -it --entrypoint=/busybox/sh gcr.io/kaniko-project/executor:debug

Taking snapshot of full filesystem…

日志报错误 Taking snapshot of full filesystem...,可以尝试添加参数(参考):

  • --compressed-caching=false

kaniko 不支持 Docker 中存在正则表达式

参考

配置 proxy

参考

        --build-arg "http_proxy=${http_proxy}"
        --build-arg "HTTP_PROXY=${http_proxy}"
        --build-arg "https_proxy=${https_proxy}"
        --build-arg "HTTPS_PROXY=${https_proxy}"

参考

  1. https://github.com/GoogleContainerTools/kaniko
最新评论
加载中...
Home Archives Categories Tags Statistics