Kubebuilder是由kubernetes-sigs维护的，基于 k8s 控制器运行时封装的 k8s Operator 主流开发工具
介绍
Kubebuilder 的核心组件具有3个职责：
- 负责运行所有的 Controllers
- 初始化共享 caches，包含
listAndWatch 功能
- 初始化 clients 用于与 kubernetes ApiServer 通信
一些关键字：
- GV: GroupVersion
- GVK: GroupVersionKind
- GVR: GroupVersionResource
- API Group：是相关API功能的集合，每个 Group 拥有一或多个 Versions，用于接口的演进
- Kinds: 每个GV都包含的多个API类型
- Resource 是 Kind 的对象标识（resource type）
- Finalizer：在一般情况下，如果资源被删除之后，我们虽然能够被触发删除事件，但是这个时候从Cache里面无法读取任何被删除对象的信息，这样一来导致很多垃圾清理工作因为信息不足无法进行，K8s的Finalizer字段用于处理这种情况
开发环境
- docker-ce 20.10.+
- go
- v1.15+ (kubebuilder v3.0 < v3.1)
- v1.16+ (kubebuilder v3.1 < v3.3)
- v1.17+ (kubebuilder v3.3+)
- kubectl 1.22.0+
- kubernetes v1.22.0+ cluster，参考：Kubernetes 安装部署
安装
curl -L -o kubebuilder "https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)"
chmod +x kubebuilder && mv kubebuilder /usr/local/bin/
安装后，使用 kubebuilder -h 查看命令帮助
生成代码
示例的代码见：https://github.com/kbcx/xca-operator
# mkdir -p xca-operator
# cd xca-operator
# $ kubebuilder init --domain kb.cx --project-name xca-operator --fetch-deps false --repo github.com/kbcx/xca-operator --owner xiexianbin
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.12.2
Update dependencies:
$ go mod tidy
Next: define a resource with:
$ kubebuilder create api
init 时默认参数 --plugins=""go.kubebuilder.io/v3 --project-version=3，使用 kubebuilder help init 查看 init 命令详情：
$ kubebuilder help init
Initialize a new project including the following files:
- a "go.mod" with project dependencies
- a "PROJECT" file that stores project configuration
- a "Makefile" with several useful make targets for the project
- several YAML files for project deployment under the "config" directory
- a "main.go" file that creates the manager that will run the project controllers
...
- 创建 API，生成 CRD 和 Controller
# kubebuilder create api -h
# kubebuilder create api --group xca --version v1alpha1 --kind Xtls
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1alpha1/xtls_types.go
controllers/xtls_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin
test -s /Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen || GOBIN=/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.9.2
/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests
说明：
# kubebuilder create webhook --group xca --version v1alpha1 --kind Xtls --defaulting --programmatic-validation
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1alpha1/xtls_webhook.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new Webhook and generate the manifests with:
$ make manifests
go mod tidy
$ tree ../xca-operator
../xca-operator
├── Dockerfile # 制作镜像的 Dockerfile
├── LICENSE
├── Makefile
├── PROJECT
├── README.md
├── api
│ └── v1alpha1 # API 资源，Resource 对象的相关信息
│ ├── groupversion_info.go
│ ├── webhook_suite_test.go
│ ├── xtls_types.go # 自定义CRD类型
│ ├── xtls_webhook.go
│ └── zz_generated.deepcopy.go
├── bin
│ ├── controller-gen
│ ├── kustomize
│ ├── manager
│ └── setup-envtest
├── config
│ ├── certmanager
│ │ ├── certificate.yaml
│ │ ├── kustomization.yaml
│ │ └── kustomizeconfig.yaml
│ ├── crd # 注册到 K8S 的自定义资源（CRD）的描述文件（YAML）
│ │ ├── bases
│ │ │ └── xca.kb.cx_xtls.yaml
│ │ ├── kustomization.yaml
│ │ ├── kustomizeconfig.yaml
│ │ └── patches
│ │ ├── cainjection_in_xtls.yaml
│ │ └── webhook_in_xtls.yaml
│ ├── default
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ ├── manager_config_patch.yaml
│ │ ├── manager_webhook_patch.yaml
│ │ └── webhookcainjection_patch.yaml
│ ├── manager # 部署到K8S的 manager YAML
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── prometheus
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ ├── rbac # 部署到K8S的 rbac YAML
│ │ ├── auth_proxy_client_clusterrole.yaml
│ │ ├── auth_proxy_role.yaml
│ │ ├── auth_proxy_role_binding.yaml
│ │ ├── auth_proxy_service.yaml
│ │ ├── kustomization.yaml
│ │ ├── leader_election_role.yaml
│ │ ├── leader_election_role_binding.yaml
│ │ ├── role.yaml
│ │ ├── role_binding.yaml
│ │ ├── service_account.yaml
│ │ ├── xtls_editor_role.yaml
│ │ └── xtls_viewer_role.yaml
│ ├── samples # CRD 的样例文件，可以通过它直接 apply 到 k8s 集群中
│ │ └── xca_v1alpha1_xtls.yaml
│ └── webhook
│ ├── kustomization.yaml
│ ├── kustomizeconfig.yaml
│ ├── manifests.yaml
│ └── service.yaml
├── controllers # Controller 代码
│ ├── suite_test.go
│ └── xtls_controller.go # 协调自定义资源终态的控制逻辑
├── go.mod
├── go.sum
├── hack
│ └── boilerplate.go.txt
├── main.go # 函数入口
├── utils
│ ├── utils.go
│ └── utils_test.go
└── vendor
...
792 directories, 3800 files
安装
make manifests 重新生成修改后的 CRD 描述make generate 重新生成代码- 开发阶段，可以
本地运行 controller
make run
make docker-build
make docker-build docker-push IMG=xiexianbin/xca-operator:latest
make install 安装 CRDs 到 k8s 环境
- 默认选择
~/.kube/config 指定的 k8s 环境
- 采用
kubectl get crd 查看自定义资源是否安装到指定的 k8s 环境
make uninstall 卸载 CRDs
验证 CRD
config/samples/xca_v1alpha1_xtls.yaml
apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
name: xtls-sample
spec:
# TODO(user): Add fields here
kubectl apply -f config/samples/xca_v1alpha1_xtls.yaml
kubectl get Xca
kubectl get Xca -o yaml
kubectl delete -f config/samples/xca_v1alpha1_xtls.yaml
实操日志
$ kubectl apply -f config/samples/xca_v1alpha1_xtls.yaml
xtls.xca.kb.cx/xtls-sample created
$ kubectl get xtls
NAME AGE
xtls-sample 16s
$ kubectl get xtls -o yaml
apiVersion: v1
items:
- apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
creationTimestamp: "2022-09-03T04:07:58Z"
generation: 1
name: xtls-sample
namespace: default
resourceVersion: "708034"
uid: 992ab4e9-ace8-447d-93f7-1ccdf6f648d0
spec:
cn: xiexianbin.cn
days: 365
domains:
- xiexianbin.cn
- www.xiexianbin.cn
ips:
- 1.1.1.1
- 127.0.0.1
keyBits: 2048
kind: List
metadata:
resourceVersion: ""
$ kubectl get xtls xtls-sample -o yaml
apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
creationTimestamp: "2022-09-03T04:07:58Z"
generation: 1
name: xtls-sample
namespace: default
resourceVersion: "708034"
uid: 992ab4e9-ace8-447d-93f7-1ccdf6f648d0
spec:
cn: xiexianbin.cn
days: 365
domains:
- xiexianbin.cn
- www.xiexianbin.cn
ips:
- 1.1.1.1
- 127.0.0.1
keyBits: 2048
root@k8s-master:~# kubectl get xtls.xca.kb.cx
NAME AGE
xtls-sample 75m
root@k8s-master:~# kubectl get xtls.xca.kb.cx -o yaml
apiVersion: v1
items:
- apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
creationTimestamp: "2022-09-13T13:45:51Z"
generation: 1
name: xtls-sample
namespace: default
resourceVersion: "748371"
uid: 8744e36a-5e95-4b44-9926-724c2f53703b
spec:
cn: xiexianbin.cn
days: 365
domains:
- xiexianbin.cn
- www.xiexianbin.cn
ips:
- 1.1.1.1
- 127.0.0.1
keyBits: 2048
status:
active: true
lastUpdateTime: "2022-09-13T13:45:51Z"
kind: List
metadata:
resourceVersion: ""
root@k8s-master:~# kubectl get secrets xtls-sample -o yaml
apiVersion: v1
data:
tls.crt: xxx
tls.key: xxx
kind: Secret
metadata:
creationTimestamp: "2022-09-13T13:45:23Z"
labels:
createAtms: "1663076722870"
creatorBy: xca-operator
name: xtls-sample
namespace: default
resourceVersion: "748326"
uid: 0b277c5a-4e69-4375-b39c-4e69a9230c59
type: kubernetes.io/tls