skopeo 是由 containers 提供远程仓库的镜像管理能力，包括：复制镜像、检索镜像信息、删除镜像、从公共镜像仓库向私有仓库同步镜像等
介绍
Skopeo 支持的镜像仓库类型：
- containers-storage
- dir
- docker://docker：Docker Registry HTTP API V2
- docker-archive:path[:docker-reference]：docker save 打包的镜像
- docker-daemon:docker-reference：本地的 docker 镜像
- oci:path:tag
支持的镜像格式：
dir:/PATHdocker://k8s.gcr.io/pause:3.2docker-daemon:alpine:latestdocker-archive:alpine.taroci:oci:alpine:latest
安装
sudo yum -y install skopeo
brew install skopeo
# Debian Bullseye, Testing or Unstable/Sid
sudo apt-get update
sudo apt-get -y install skopeo
# Ubuntu 20.10 and newer
sudo apt-get -y update
sudo apt-get -y install skopeo
# Ubuntu 20.04
. /etc/os-release
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
# https://download.opensuse.org/repositories/home:/alvistack/xUbuntu_20.04/amd64/
curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/Release.key" | sudo apt-key add -
sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get -y install skopeo
# install go
wget https://go.dev/dl/go1.17.7.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.17.7.linux-amd64.tar.gz
# 配置环境变量 vim /etc/profile
export GOROOT=/usr/local/go
export PATH=$PATH:$GOROOT/bin
export GOPATH=/usr/local/gopath
source /etc/profile
# install skopeo
sudo apt install libgpgme-dev libassuan-dev libbtrfs-dev libdevmapper-dev pkg-config gcc make
git clone https://github.com/containers/skopeo $GOPATH/src/github.com/containers/skopeo
cd $GOPATH/src/github.com/containers/skopeo
git checkout v1.6.1
# build skopeo
make bin/skopeo
参考：https://github.com/containers/skopeo/blob/main/install.md
源码打包
help
$ skopeo -h
Various operations with container images and container image registries
Usage:
skopeo [command]
Available Commands:
copy Copy an IMAGE-NAME from one location to another
delete Delete image IMAGE-NAME
help Help about any command
inspect Inspect image IMAGE-NAME
list-tags List tags in the transport/repository specified by the REPOSITORY-NAME
login Login to a container registry
logout Logout of a container registry
manifest-digest Compute a manifest digest of a file
standalone-sign Create a signature using local files
standalone-verify Verify a signature using local files
sync Synchronize one or more images from one location to another
Flags:
--command-timeout duration timeout for the command execution
--debug enable debug output
-h, --help help for skopeo
--insecure-policy run the tool without any policy check
--override-arch ARCH use ARCH instead of the architecture of the machine for choosing images
--override-os OS use OS instead of the running OS for choosing images
--override-variant VARIANT use VARIANT instead of the running architecture variant for choosing images
--policy string Path to a trust policy file
--registries.d DIR use registry configuration files in DIR (e.g. for container signature storage)
--tmpdir string directory used to store temporary files
-v, --version Version for Skopeo
Use "skopeo [command] --help" for more information about a command.
使用
镜像信息
$ skopeo inspect docker://docker.io/alpine
{
"Name": "docker.io/library/alpine",
"Digest": "sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300",
"RepoTags": [
"2.6",
...
"3.9.5",
"3.9.6",
"edge",
"latest"
],
"Created": "2021-11-24T20:19:40.483367546Z",
"DockerVersion": "20.10.7",
"Labels": null,
"Architecture": "amd64",
"Os": "linux",
"Layers": [
"sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3"
],
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
]
}
$ skopeo inspect docker://docker.io/alpine:latest | jq '.Digest'
"sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300"
$ skopeo list-tags docker://alpine
复制镜像
skopeo 可以从多种存储复制镜像：
- Container registries
- Container Storage backends
- Local directories
- Local OCI-layout directories
$ skopeo copy docker://quay.io/buildah/stable docker://registry.internal.company.com/buildah
$ skopeo copy oci:busybox_ocilayout:latest dir:existingemptydirectory
$ skopeo copy docker://ubuntu:20.04 dir:/root/ubuntu-20.04
$ skopeo copy docker://k8s.gcr.io/pause:3.2 docker://docker.io/gcmirrors/pause:3.2 --dest-authfile /root/.docker/config.json
删除镜像
$ skopeo delete docker://localhost:5000/imagename:latest
同步仓库
$ skopeo sync --src docker --dest dir registry.example.com/busybox /media/usb
$ skopeo sync --src gcr --dest docker gcr docker://gcmirrors
$ skopeo sync --insecure-policy --src-tls-verify=false --dest-tls-verify=false --src docker --dest docker gcr.io/google-containers/cadvisor docker.io/gcmirrors
认证
$ skopeo login --username USER myregistrydomain.com:5000
$ skopeo inspect --creds=testuser:testpassword docker://myregistrydomain.com:5000/busybox
$ skopeo copy --src-creds=testuser:testpassword docker://myregistrydomain.com:5000/private oci:local_oci_image
$ skopeo logout ...
ssl 规避
skopeo ... --insecure-policy --src-tls-verify=false --dest-tls-verify=false
镜像格式探索
oci 镜像格式
# 复制docker镜像为OCI格式
$ skopeo copy docker://alpine:latest oci:alpine:latest
Getting image source signatures
Copying blob df9b9388f04a done
Copying config 13278536a0 done
Writing manifest to image destination
Storing signatures
# 目录树查看
$ tree alpine/
alpine/
├── blobs
│ └── sha256
│ ├── 13278536a0c1e9ace25e40a736808e9fb711b9521757eb57769dc30fd2cb1b15
│ ├── 167089c8e895bba63d2245845c32d555689ddb7bb1c05af107ef0d3ff92ec5ae
│ └── df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139
├── index.json
└── oci-layout
2 directories, 5 files
# 查看镜像的RooTFS
$ docker pull alipne:latest
$ docker inspect alpine:latest --format "{{.RootFS}}"
{layers [sha256:4fc242d58285699eca05db3cc7c7122a2b8e014d9481f323bd9277baacfa0628] }
# OCI 镜像版本
$ cat alpine/oci-layout | jq .
{
"imageLayoutVersion": "1.0.0"
}
# 镜像入口文件
$ cat alpine/index.json | jq .
{
"schemaVersion": 2,
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json", # 文件格式
"digest": "sha256:167089c8e895bba63d2245845c32d555689ddb7bb1c05af107ef0d3ff92ec5ae", # 文件sha256摘要，在 blobs/sha256 文件目录中有对应的文件
"size": 348, # 大小
"annotations": {
"org.opencontainers.image.ref.name": "latest" # tag 版本
}
}
]
}
# 查看文件
$ cat alpine/blobs/sha256/167089c8e895bba63d2245845c32d555689ddb7bb1c05af107ef0d3ff92ec5ae | jq .
{
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json", # 配置信息
"digest": "sha256:13278536a0c1e9ace25e40a736808e9fb711b9521757eb57769dc30fd2cb1b15",
"size": 585
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", # 镜像文件
"digest": "sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139", # 镜像文件
"size": 2814559
}
]
}
# 配置信息说明镜像的格式、rootfs、Cmd等
$ cat alpine/blobs/sha256/13278536a0c1e9ace25e40a736808e9fb711b9521757eb57769dc30fd2cb1b15 | jq .
{
"created": "2022-04-05T00:19:59.912662499Z",
"architecture": "amd64",
"os": "linux",
"config": {
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
]
},
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:4fc242d58285699eca05db3cc7c7122a2b8e014d9481f323bd9277baacfa0628"
]
},
"history": [ # 显示每层镜像的构建历史
{
"created": "2022-04-05T00:19:59.790636867Z",
"created_by": "/bin/sh -c #(nop) ADD file:5d673d25da3a14ce1f6cf66e4c7fd4f4b85a3759a9d93efb3fd9ff852b5b56e4 in / "
},
{
"created": "2022-04-05T00:19:59.912662499Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
}
]
}
# 解压文件
$ mkdir -p rootfs/01
$ tar -xf blobs/sha256/df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139 -C rootfs/01
$ ls rootfs/01/
bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
解压后的oci镜像格式
- 安装加压工具 umoci（修改Open Container images的工具）
apt install umoci
$ umoci --help
NAME:
umoci - umoci modifies Open Container images
USAGE:
umoci [global options] command [command options] [arguments...]
VERSION:
unknown
AUTHOR:
Aleksa Sarai <asarai@suse.com>
COMMANDS:
raw advanced internal image tooling
help, h Shows a list of commands or help for one command
image:
config modifies the image configuration of an OCI image
unpack unpacks a reference into an OCI runtime bundle
repack repacks an OCI runtime bundle into a reference
new creates a blank tagged OCI image
tag creates a new tag in an OCI image
remove, rm removes a tag from an OCI image
stat displays status information of an image manifest
insert insert content into an OCI image
layout:
gc garbage-collects an OCI image's blobs
init create a new OCI layout
list, ls lists the set of tags in an OCI layout
GLOBAL OPTIONS:
--verbose alias for --log=info
--log value set the log level (debug, info, [warn], error, fatal) (default: "warn")
--help, -h show help
--version, -v print the version
$ skopeo copy docker://alpine:latest oci:alpine:latest
# 解压
$ umoci unpack --image alpine:latest alpine-bundle
# 查看目录树
$ tree alpine-bundle/ -L 1
alpine-bundle/
├── config.json # 镜像定义文件，包括 ociVersion、rootfs、主机名、环境变量、挂载信息等
├── rootfs # 跟文件夹
├── sha256_167089c8e895bba63d2245845c32d555689ddb7bb1c05af107ef0d3ff92ec5ae.mtree
└── umoci.json
1 directory, 3 files
# umoci 配置文件
$ cat alpine-bundle/umoci.json | jq .
{
"umoci_version": "2",
"from_descriptor_path": {
"descriptor_walk": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:167089c8e895bba63d2245845c32d555689ddb7bb1c05af107ef0d3ff92ec5ae",
"size": 348,
"annotations": {
"org.opencontainers.image.ref.name": "latest"
}
}
]
},
"map_options": {
"uid_mappings": null,
"gid_mappings": null,
"rootless": false
}
}
$ cd alpine-bundle
$ runc run sh
/ # ls
bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr
...
删除文件演示
实现参考：whiteouts.go
FROM alpine:latest
RUN mkdir -p /foo/bar && echo "hello" >> /foo/bar/1.txt
RUN echo "world" >> /foo/bar/1.txt
RUN rm /foo/bar/1.txt
RUN rm -rf /foo
docker build . -t xiexianbin/alpine:show-delete
$ skopeo copy docker://xiexianbin/alpine:show-delete oci:alpine:show-delete
$ tree alpine/
alpine/
├── blobs
│ └── sha256
│ ├── 05e2973296e31366cb5528319cd717ce5c71aeb744d7351d6219dbebea6e5e64
│ ├── 289d56326069f2e51bdfa0fc479ea093158d3695a7eee8c7aa57f9308a20108b
│ ├── 534f263a339c13c19847e369b7595234722d0b09f8ca481dc0d5b2e3ed907014
│ ├── 7708d16f9641890aa7813dddf54a40b2935647ec1405bf42da4e8f7bfb85fa19
│ ├── cdb468907b4890d83a9af8f94cf3e969c1e7f1f8f278ff777f5566348ef09547
│ ├── dc16c7120c5b3d953442fde714b5dda32a4612d13b15cbd1d3e49a785916d2a8
│ └── df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139
├── index.json
└── oci-layout
2 directories, 9 files
$ cat alpine/index.json | jq .
{
"schemaVersion": 2,
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:cdb468907b4890d83a9af8f94cf3e969c1e7f1f8f278ff777f5566348ef09547",
"size": 964,
"annotations": {
"org.opencontainers.image.ref.name": "show-delete"
}
}
]
}
$ cat alpine/blobs/sha256/cdb468907b4890d83a9af8f94cf3e969c1e7f1f8f278ff777f5566348ef09547 | jq .
{
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:534f263a339c13c19847e369b7595234722d0b09f8ca481dc0d5b2e3ed907014",
"size": 1312
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139",
"size": 2814559
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:dc16c7120c5b3d953442fde714b5dda32a4612d13b15cbd1d3e49a785916d2a8",
"size": 178
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:05e2973296e31366cb5528319cd717ce5c71aeb744d7351d6219dbebea6e5e64",
"size": 160
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:289d56326069f2e51bdfa0fc479ea093158d3695a7eee8c7aa57f9308a20108b",
"size": 148
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:7708d16f9641890aa7813dddf54a40b2935647ec1405bf42da4e8f7bfb85fa19",
"size": 93
}
]
}
$ mkdir rootfs rootfs/02 rootfs/03 rootfs/04 rootfs/05
# 第一层：为alpine的镜像，不查看了
# 第二层
$ tar -xf alpine/blobs/sha256/dc16c7120c5b3d953442fde714b5dda32a4612d13b15cbd1d3e49a785916d2a8 -C rootfs/02/
$ tree -alh rootfs/02/
rootfs/02/
└── [4.0K] foo
├── [4.0K] bar
│ └── [ 6] 1.txt
└── [ 0] .wh..wh..opq
2 directories, 2 files
$ cat rootfs/02/foo/bar/1.txt
hello
# 第三层
$ tar -xf alpine/blobs/sha256/05e2973296e31366cb5528319cd717ce5c71aeb744d7351d6219dbebea6e5e64 -C rootfs/03
$ tree -alh rootfs/03/
rootfs/03/
└── [4.0K] foo
└── [4.0K] bar
└── [ 12] 1.txt
2 directories, 1 file
$ cat rootfs/03/foo/bar/1.txt
hello
world
# 第四层
$ tar -xf alpine/blobs/sha256/289d56326069f2e51bdfa0fc479ea093158d3695a7eee8c7aa57f9308a20108b -C rootfs/04
$ tree -alh rootfs/04/
rootfs/04/
└── [4.0K] foo
└── [4.0K] bar
└── [ 0] .wh.1.txt # 删除 1.txt 文件
2 directories, 1 file
# 第五层
$ tar -xf alpine/blobs/sha256/7708d16f9641890aa7813dddf54a40b2935647ec1405bf42da4e8f7bfb85fa19 -C rootfs/05/
$ tree -alh rootfs/05/
rootfs/05/
└── [ 0] .wh.foo # 删除 foo 文件夹
0 directories, 1 file
扩展
基于 skopeo 开发 python3-cisctl 结合 Github Actions 实现镜像仓库(如从 google container register 到 docker hub)的同步，同步镜像查询 https://mirrors.kb.cx。
F&Q
python3-cisctl 在每天同步镜像时，总是由于镜像摘要不对反复同步，经查是 mediaType 版本不对导致的，好像还没好的解决方法，参考，错误的mediaType示例如下：
...
"sha256:4fe91b03ed0b3c02b0833e78d17a7e92336a864b614d70cc48bc8dcb955441d3":{
"imageSizeBytes":"14824751",
"layerId":"",
"mediaType":"application/vnd.docker.distribution.manifest.v2+json",
"tag":[
],
"timeCreatedMs":"0",
"timeUploadedMs":"1645827416725"
},
...
"name":"knative-releases/knative.dev/serving/cmd/webhook",
{
"count":82,
"next":"https://registry.hub.docker.com/v2/repositories/gcrioknative/serving-webhook/tags?n=10\u0026page=2",
...
{
"creator":17383423,
"id":285458617,
"images":[],
"last_updated":"2022-11-23T05:17:47.464209Z",
"last_updater":17383423,
"last_updater_username":"xmirrors",
"name":"v1.7.1",
"repository":16804784,
"full_size":15865602,
"v2":true,
"tag_status":"active",
"tag_last_pulled":null,
"tag_last_pushed":"2022-11-23T05:17:47.464209Z",
"media_type":"application/vnd.docker.container.image.v1+json",
"digest":"sha256:a588b4df061c953aced32fda29f484d07be9d121cd14b29e18220a92cf49d9ff"
},