Velero是由Vmware捐献给CNCF的，对Kubernetes进行备份、还原和迁移的工具。
介绍
Velero是一个云原生的灾难备份、还原和迁移工具，使用Golang开发、且开源，用于安全的备份、还原和迁移 Kubernetes 集群资源数据（etcd数据）。
参考
说明：
使用
创建准备
该账户用于通过调用 k8s API 查询资源，参考：
- 参考 cfssl 证书签发工具使用 为 velero 签发证书
- k8s API 认证 kubeconf 文件生产
# 生成认证文件
$ export KUBE_APISERVER="https://172.20.0.241:6443"
$ kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.crt \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=./velero.kubeconfig
# 设置客户端证书
$ kubectl config set-credentials velero \
--client-certificate=./velero.pem \
--client-key=./velero-key.pem \
--embed-certs=true \
--kubeconfig=./velero.kubeconfig
# 设置上下文参数
$ kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=velero \
--namespace=velero-system \
--kubeconfig=./velero.kubeconfig
- 设置默认上下文
$ kubectl config user-context kubernetes --kubeconfig=./velero.kubeconfig
- 创建 namespace
kubectl create ns velero-system
- 在 k8s 集群中创建 velero 用户
kubectl create clusterrolebinding velero --clusterrole=cluster-admin --user=velero
- 切换默认上下文
root@k8s-master:~/velero# kubectl config use-context kubernetes --kubeconfig velero.kubeconfig
Switched to context "kubernetes".
安装 minio
- minio 提供 S3 接口，用来保存 velero 备份文件
- 参考 MinIO 介绍
- 也可以使用安装包里的 dp 快速创建
velero-v1.9.0-linux-amd64/examples/minio/00-minio-deployment.yaml
- 创建 aws 认证文件：
minio-auth.txt
$ cat minio-auth.txt
[default]
aws_access_key_id = admin
aws_secret_access_key = minioadmin
安装 velero
安装 velero 客户端
wget https://github.com/vmware-tanzu/velero/releases/download/v1.9.0/velero-v1.9.0-linux-amd64.tar.gz
tar -zxvf velero-v1.9.0-linux-amd64.tar.gz # 有示例文件可参考
cp velero-v1.9.0-linux-amd64/velero /usr/local/bin
$ velero --help
velero --help
Velero is a tool for managing disaster recovery, specifically for Kubernetes
cluster resources. It provides a simple, configurable, and operationally robust
way to back up your application state and associated data.
If you're familiar with kubectl, Velero supports a similar model, allowing you to
execute commands such as 'velero get backup' and 'velero create schedule'. The same
operations can also be performed as 'velero backup get' and 'velero schedule create'.
Usage:
velero [command]
Available Commands:
backup Work with backups
backup-location Work with backup storage locations
bug Report a Velero bug
client Velero client related commands
completion Generate completion script
create Create velero resources
debug Generate debug bundle
delete Delete velero resources
describe Describe velero resources
get Get velero resources
help Help about any command
install Install Velero
plugin Work with plugins
restic Work with restic
restore Work with restores
schedule Work with schedules
snapshot-location Work with snapshot locations
uninstall Uninstall Velero
version Print the velero version and associated image
安装 velero 服务端
$ velero install \
--kubeconfig ./velero.kubeconfig \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.5.0 \
--bucket velero \
--secret-file ./minio-auth.txt \
--use-volume-snapshots=false \
--namespace velero-system \
--backup-location-config region=minio,s3ForcePathStyle="true",s3Url="http://172.20.0.241:9000"
安装日志：
root@k8s-master:~/velero# velero install --kubeconfig ./velero.kubeconfig --provider aws --plugins velero/velero-plugin-for-aws:v1.5.0 --bucket velero --secret-file ./minio-auth.txt --use-volume-snapshots=false --namespace velero-system --backup-location-config region=minio,s3ForcePathStyle="true",s3Url="http://172.20.0.241:9000"
CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: attempting to create resource client
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource client
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource client
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource client
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource client
CustomResourceDefinition/resticrepositories.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: attempting to create resource client
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: attempting to create resource client
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource client
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource client
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero-system: attempting to create resource
Namespace/velero-system: attempting to create resource client
Namespace/velero-system: already exists, proceeding
Namespace/velero-system: created
ClusterRoleBinding/velero-velero-system: attempting to create resource
ClusterRoleBinding/velero-velero-system: attempting to create resource client
ClusterRoleBinding/velero-velero-system: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: attempting to create resource client
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: attempting to create resource client
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: attempting to create resource client
BackupStorageLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: attempting to create resource client
Deployment/velero: created
Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero-system' to view the status.
默认使用的镜像是：velero/velero:v1.9.0
$ kubectl -n velero-system get pod
NAME READY STATUS RESTARTS AGE
velero-96bbcf5fc-xgzrq 1/1 Running 0 8m52s
使用
备份
$ DATE=`date +%Y%m%d%H%M%S`
$ velero backup create \
istio-backup-${DATE} \
--namespace velero-system \
--include-namespaces istio-system \
--kubeconfig ./velero.kubeconfig
Backup request "istio-backup-20220901215842" submitted successfully.
Run `velero backup describe istio-backup-20220901215842` or `velero backup logs istio-backup-20220901215842` for more details.
$ kubectl -n velero-system get backups.velero.io
NAME AGE
istio-backup-20220901215842 98s
$ velero -n velero-system backup describe istio-backup-20220901215842
Name: istio-backup-20220901215842
Namespace: velero-system
Labels: velero.io/storage-location=default
Annotations: velero.io/source-cluster-k8s-gitversion=v1.25.0
velero.io/source-cluster-k8s-major-version=1
velero.io/source-cluster-k8s-minor-version=25
Phase: Completed
Errors: 0
Warnings: 0
Namespaces:
Included: istio-system
Excluded: <none>
Resources:
Included: *
Excluded: <none>
Cluster-scoped: auto
Label selector: <none>
Storage Location: default
Velero-Native Snapshot PVs: auto
TTL: 720h0m0s
Hooks: <none>
Backup Format Version: 1.1.0
Started: 2022-09-01 21:57:47 +0800 CST
Completed: 2022-09-01 21:57:51 +0800 CST
Expiration: 2022-10-01 21:57:47 +0800 CST
Total items to be backed up: 77
Items backed up: 77
Velero-Native Snapshots: <none included>
velero -n velero-system backup logs istio-backup-20220901215842
- 登录 Minio 可以查看到对应的备份
http://172.20.0.241:9001/buckets/velero/browse
还原
$ velero restore create \
--namespace velero-system \
--kubeconfig ./velero.kubeconfig \
--from-backup istio-backup-20220901215842 --wait
Restore request "istio-backup-20220901215842-20220901220800" submitted successfully.
Waiting for restore to complete. You may safely press ctrl-c to stop waiting - your restore will continue in the background.
.............
Restore completed with status: Completed. You may check for more information using the commands `velero restore describe istio-backup-20220901215842-20220901220800` and `velero restore logs istio-backup-20220901215842-20220901220800`.
$ velero -n velero-system restore logs istio-backup-20220901215842-20220901220800