Kubernetes 提供多种认证机制，本文详细介绍。
概念
扩展：HTTP 认证介绍
Kubernetes 认证机制如下：
- 身份认证（Authentication）通过认证插件实现，检查用户是否为合法用户，认证包括：
- 令牌认证 token（Bootstrap Tokens），通过预共享密钥
- JWT tokens
- TLS 认证，通过 X509 client certs 证书 Subject 添加：
- CN: user/serviceaccount
- O: group
- user/password
- Static Token File
- Static Password File
- Service Account Tokens
- OpenId Connect Tokens
- …
- 授权（Authorization）：通过授权插件实现，判断该用户是否具有该操作的权限
- RBAC（Role-based Access Controller），许可授权，RBAC 为主流方式
- ABAC
- Node 认证
- Webhook 认证
- 准入机制（Admission Control）：请求的最后一个步骤，一般用于拓展功能，如检查 pod 的 resource 是否配置，yaml 配置的安全是否合规等。一般使用 admission webhooks 来实现，参考
MutatingAdmissionWebhook 修改配置ValidatingAdmissionWebhook 验证配置- 需要在
kube-apiserver 启动参数添加 - --enable-admission-plugins=MutatingAdmissionWebhook,ValidatingAdmissionWebhook
Kubernetes 的资源的权限分配，通过用户扮演角色来实现：
- 命名空间（namespace）资源，通过 RoleBinding 实现权限分配
- Role（角色），也可以表示为 permission（许可） = Operations + Objects
- RoleBinding
- UserAccount
- Group
- ServiceAccount（若采用 key 认证，通过 Subject 字段指定）
- 创建Pod时，指定
spec.serviceAccountName
- Role
- 集群资源，通过 ClusterRole, ClusterRoleBinding 实现权限分配
- Kubernetes 资源包括：
- resource group
- resource
- non-resource url
其中，可以通过 RoleBinding 绑定 ClusterRole 实现用户对指定命名空间权限限制。
关于接口调用，参考：Kubernetes API 介绍
Service Account
kubectl create serviceaccount -h
kubectl get sa
kubectl create serviceaccount NAME [--dry-run=server|client|none] [options]
kubectl create serviceaccount mysa -o yaml --dry-run=client
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: null
name: mysa
$ kubectl apply -f serviceaccount-demo.yaml
serviceaccount/mysa created
$ kubectl get sa
NAME SECRETS AGE
default 1 24d
mysa 1 3s
$ kubectl get sa mysa -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"creationTimestamp":null,"name":"mysa","namespace":"default"}}
creationTimestamp: "2022-03-19T02:11:53Z"
name: mysa
namespace: default
resourceVersion: "1198052"
uid: 5eb600e0-57d4-428f-9ee8-e0d4d9b9ec6c
secrets:
- name: mysa-token-hnklw
$ kubectl get secrets mysa-token-hnklw
NAME TYPE DATA AGE
mysa-token-hnklw kubernetes.io/service-account-token 3 71s
$ kubectl describe sa mysa
Name: mysa
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: mysa-token-hnklw
Tokens: mysa-token-hnklw
Events: <none>
说明：
- 会自动创建对应的
ServiceAccount 和 Secrets
- 默认情况下存在名字为 default 的
ServiceAccount，默认注入所有容器内，默认挂载在 Pod 的 /var/run/secrets/kubernetes.io/serviceaccount/ 目录
- 创建容器时，可以通过
kubectl explain pods.spec.serviceAccountName 指定 Service Account，然后通过 ServiceAccount 和 role 的 banding 赋予不同的权限
- 可以为 ServiceAccount bingding 对应的
Image pull secrets 赋予拉去镜像的认证信息，该方法比创建容器是指定 kubectl explain pods.spec.imagePullSecrets 更为方便
创建用户PKI
kubelet 认证机制
默认通过加载 ~/.kube/config 中的认证信息进行认证，我们可以使用 kubectl config 进行配置，示例：
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.20.0.81:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
我们可以在控制节点的 /etc/kubernetes/pki/ 目录下发现对应的 CA key。
创建用户
下面为用户 xianbin 创建认证的 key，也可以参考 cfssl 证书签发工具使用
cd /etc/kubernetes/pki/
umask 077; openssl genrsa -out xianbin.key 2048
openssl req -new -key xianbin.key -out xianbin.csr -subj "/CN=xianbin"
其中，Subject 中：
openssl x509 -req -in xianbin.csr -CA ./ca.crt -CAkey ./ca.key -CAcreateserial -out xianbin.crt -days 700
$ openssl x509 -in xianbin.crt -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
6c:ae:6f:1d:78:0f:0c:90:57:dd:4f:78:14:0b:d1:61:bc:92:7c:1c
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = kubernetes # 签署者
Validity
Not Before: Mar 19 02:38:46 2022 GMT
Not After : Feb 17 02:38:46 2024 GMT
Subject: CN = xianbin # 被签署的用户
...
$ kubectl config set-credentials -h
kubectl config set-credentials NAME [--client-certificate=path/to/certfile]
[--client-key=path/to/keyfile] [--token=bearer_token] [--username=basic_user]
[--password=basic_password] [--auth-provider=provider_name] [--auth-provider-arg=key=value]
[--exec-command=exec_command] [--exec-api-version=exec_api_version] [--exec-arg=arg]
[--exec-env=key=value] [options]
$ kubectl config set-credentials xianbin --client-certificate=./xianbin.crt --client-key=./xianbin.key --embed-certs=true
User "xianbin" set.
$ kubectl config view
...
users:
...
- name: xianbin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
$ kubectl config set-context xianbin@kubernetes --cluster=kubernetes --user=xianbin
Context "xianbin@kubernetes" created.
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.20.0.81:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context:
cluster: kubernetes
user: xianbin
name: xianbin@kubernetes # 新设置的 context
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: xianbin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
$ kubectl config use-context xianbin@kubernetes
Switched to context "xianbin@kubernetes".
kubectl config set-cluster -h
kubectl config set-cluster NAME [--server=server]
[--certificate-authority=path/to/certificate/authority] [--insecure-skip-tls-verify=true]
[--tls-server-name=example.com] [options]
- 命令执行，默认情况下，当前用户 xianbin 没有任何权限
xiexianbin@k8s-master:~$ kubectl get pod
Error from server (Forbidden): pods is forbidden: User "xianbin" cannot list resource "pods" in API group "" in the namespace "default"
xiexianbin@k8s-master:~$ kubectl get pod -n kube-system
Error from server (Forbidden): pods is forbidden: User "xianbin" cannot list resource "pods" in API group "" in the namespace "kube-system"
Role
kubectl create role -h
kubectl create role NAME --verb=verb --resource=resource.group/subresource [--resource-name=resourcename] [--dry-run=server|client|none] [options]
kubectl create role pod-reader --verb=get,list,watch --resource=pods -o yaml --dry-run=client
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader
namespace: default
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
$ kubectl apply -f role-demo.yaml
role.rbac.authorization.k8s.io/pod-reader created
$ kubectl get role
NAME CREATED AT
pod-reader 2022-03-19T21:35:04Z
$ kubectl describe role pod-reader
Name: pod-reader
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch]
RoleBinding
kubectl create rolebinding -h
kubectl create rolebinding NAME --clusterrole=NAME|--role=NAME [--user=username] [--group=groupname] [--serviceaccount=namespace:serviceaccountname] [--dry-run=server|client|none] [options]
$ kubectl create rolebinding xianbin-read-pod --role=pod-reader --user=xianbin --dry-run=client -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
creationTimestamp: null
name: xianbin-read-pod
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pod-reader
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: xianbin
其中，用户 xianbin 不存在，是标识，ssl key 中的 CN
$ kubectl apply -f rolebinding-demo.yaml
rolebinding.rbac.authorization.k8s.io/xianbin-read-pod created
$ kubectl describe rolebindings xianbin-read-pod
Name: xianbin-read-pod
Labels: <none>
Annotations: <none>
Role:
Kind: Role
Name: pod-reader
Subjects:
Kind Name Namespace
---- ---- ---------
User xianbin
Role & RoleBinding 测试
kubectl config use-context xianbin@kubernetes
# 有权限
xiexianbin@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
hello-app-sf-0 1/1 Running 0 14h
hello-app-sf-1 1/1 Running 0 14h
t1 1/1 Running 1 (2d6h ago) 14h
# 无权限
xiexianbin@k8s-master:~$ kubectl get pod -n kube-system
Error from server (Forbidden): pods is forbidden: User "xianbin" cannot list resource "pods" in API group "" in the namespace "kube-system"
ClusterRole
kubectl create clusterrole -h
kubectl create clusterrole NAME --verb=verb --resource=resource.group [--resource-name=resourcename] [--dry-run=server|client|none] [options]
kubectl create clusterrole cluster-reader --verb=get,list,watch --resource=pods -o yaml --dry-run=client
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: cluster-reader
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
$ kubectl apply -f clusterrole-demo.yaml
clusterrole.rbac.authorization.k8s.io/cluster-reader created
kubectl get clusterrole cluster-reader
NAME CREATED AT
cluster-reader 2022-03-19T21:56:47Z
$ kubectl describe clusterrole cluster-reader
Name: cluster-reader
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch]
默认的 clusterrole
获取所有 clusterrole：
kubectl get clusterrole
说明：
kubectl get clusterrole admin -o yaml
ClusterRoleBinding
kubectl create clusterrolebinding -h
kubectl create clusterrolebinding NAME --clusterrole=NAME [--user=username] [--group=groupname] [--serviceaccount=namespace:serviceaccountname] [--dry-run=server|client|none] [options]
kubectl create clusterrolebinding xianbin-read-all-pods --clusterrole=cluster-reader --user=xianbin --dry-run=client -o yaml
- clusterrolebinding-demo.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: xianbin-read-all-pods
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-reader
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: xianbin
$ kubectl apply -f clusterrolebinding-demo.yaml
clusterrolebinding.rbac.authorization.k8s.io/xianbin-read-all-pods created
$ kubectl get clusterrolebindings xianbin-read-all-pods
NAME ROLE AGE
xianbin-read-all-pods ClusterRole/cluster-reader 17s
$ kubectl describe clusterrolebindings xianbin-read-all-pods
Name: xianbin-read-all-pods
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-reader
Subjects:
Kind Name Namespace
---- ---- ---------
User xianbin
kubectl get pod
kubectl get pod -n kube-system
kubectl get pod -n ingress-nginx
若删除资源没有权限：
$ kubectl -n ingress-nginx delete pod ingress-nginx-admission-create-bkrxs
Error from server (Forbidden): pods "ingress-nginx-admission-create-bkrxs" is forbidden: User "xianbin" cannot delete resource "pods" in API group "" in the namespace "ingress-nginx"
结论：clusterrolebinding 对应的 clusterrole 后，用户或组有所有命名空间的相应的权限
默认的 cluster-admin 介绍
获取 cluster-admin 配置，默认由组 system:masters 绑定到 ClusterRole cluster-admin 上，详情如下：
$ kubectl get clusterrolebinding cluster-admin -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
creationTimestamp: "2022-02-26T16:19:44Z"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: cluster-admin
resourceVersion: "149"
uid: 495f9241-9fa1-4f8d-863c-7c48a3f9acf9
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group # 组
name: system:masters
- 组
system:masters 默认配置在 apiserver-kubelet-client.crt 中，如下：
$ cd /etc/kubernetes/pki/
$ openssl x509 -in apiserver-kubelet-client.crt -text noout
$ root@k8s-master:/etc/kubernetes/pki# openssl x509 -in ./apiserver-kubelet-client.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 4641301856873790702 (0x40693772ba52bcee)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = kubernetes
Validity
Not Before: Feb 26 16:19:27 2022 GMT
Not After : Feb 26 16:19:27 2023 GMT
Subject: O = system:masters, CN = kube-apiserver-kubelet-client
- 解析 ~/.kube/config 中 kubenetes-admin 的 client-certificate-data，通过
$ echo $client-certificate-data | base64 -d >> kubeneters-admin.crt
$ openssl x509 -in kubeneters-admin.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1774077173576686618 (0x189ec9ca372adc1a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = kubernetes
Validity
Not Before: Feb 26 16:19:27 2022 GMT
Not After : Feb 26 16:19:29 2023 GMT
Subject: O = system:masters, CN = kubernetes-admin
可以看到 kubeneters-admin 和 kube-apiserver-kubelet-client 均属于 system:masters 组。
Subject 中：
RoleBinding 绑定 ClusterRole
该场景下，ClusterRole 的权限，只针对 RoleBinding 所在的名称空间生效。因此，我们可以定义通用的 ClusterRole。
下面示例，为 xianbin 用户配置 cluster-reader，结构如下：
xianbin -> rolebinding - clusterrole: cluster-reader
kubectl get clusterrolebinding xianbin-read-all-pods
kubectl delete clusterrolebinding xianbin-read-all-pods
kubectl delete rolebindings xianbin-read-pods
kubectl create rolebinding xianbin-read-pods --clusterrole=cluster-reader --user=xianbin --dry-run=client -o yaml
- rolebinding-clusterrole-demo.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
creationTimestamp: null
name: xianbin-read-pods
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-reader
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: xianbin
root@k8s-master:~/manifests# kubectl apply -f rolebinding-clusterrole-demo.yaml
rolebinding.rbac.authorization.k8s.io/xianbin-read-pods created
root@k8s-master:~/manifests# kubectl describe rolebindings xianbin-read-pods
Name: xianbin-read-pods
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-reader
Subjects:
Kind Name Namespace
---- ---- ---------
User xianbin
xiexianbin@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
hello-app-sf-0 1/1 Running 0 14h
hello-app-sf-1 1/1 Running 0 14h
t1 1/1 Running 1 (2d6h ago) 14h
xiexianbin@k8s-master:~$ kubectl get pod -n kube-system
Error from server (Forbidden): pods is forbidden: User "xianbin" cannot list resource "pods" in API group "" in the namespace "kube-system"
结论：访问 default 命名空间下的 pod 正常，其他的不正常
default 命名空间创建管理员
kubectl delete -f rolebinding-clusterrole-demo.yaml
kubectl create rolebinding default-ns-admin --clusterrole=admin --user=xianbin
xianbin 用户只有 default 命名空间的管理员权限