调度使用示例

发布时间： 更新时间： 总字数：2057 阅读时间：5m 作者： IP上海 分享 网址

Kubernetes调度使用示例

调度示例

下面通过示例介绍 Kubernetes 的默认调度策略：

  • 节点选择器：nodeSelector，nodeName
  • affinity（通过命令 kubectl explain pods.spec.affinity 查看）:
    • nodeAffinity: 节点亲和性调度
    • podAffinity: Pod亲和性调度
    • podAntiAffinity: Pod 反亲和性调度
  • 给节点打 Trains

nodeSelector 示例

  • 帮助
kubectl explain pods.spec.nodeSelector
  • pod-nodeSelector.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-ns-demo
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
  nodeSelector:
    disktype: ssd

source repo / download raw
  • 执行
# 为节点打标签
root@k8s-master:~# kubectl label nodes k8s-node-1 disktype=ssd
node/k8s-node-1 labeled
root@k8s-master:~# kubectl get nodes k8s-node-1 --show-labels
NAME         STATUS   ROLES    AGE   VERSION   LABELS
k8s-node-1   Ready    <none>   27d   v1.23.4   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=ssd,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node-1,kubernetes.io/os=linux

# 创建 Pod
root@k8s-master:~# kubectl apply -f pod-nodeSelector.yaml
pod/pod-ns-demo created

# 查看，默认调度到 k8s-node-1 节点
root@k8s-master:~# kubectl get pod -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP           NODE         NOMINATED NODE   READINESS GATES
pod-ns-demo   1/1     Running   0          57s   10.244.1.2   k8s-node-1   <none>           <none>

nodeAffinity 示例

  • 帮助
kubectl explain pods.spec.affinity.nodeAffinity

有两种配置方式：

  1. preferredDuringSchedulingIgnoredDuringExecution：软亲和性，尽量满足在相同属性的节点，不满足也能创建成功
  2. requiredDuringSchedulingIgnoredDuringExecution：硬亲和性，必须满足在相同属性的节点，不满足创建不成功
  • pod-nodeAffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-na-demo
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: zone
            operator: In
            values:
            - foo

source repo / download raw
  • pod-nodeAffinity-required 执行
# 创建
root@k8s-master:~# kubectl apply -f pod-nodeAffinity-required.yaml
pod/pod-na-demo created

# 查看
root@k8s-master:~# kubectl get pod
NAME          READY   STATUS    RESTARTS   AGE
pod-na-demo   0/1     Pending   0          4s
root@k8s-master:~# kubectl describe pod pod-na-demo
...
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  12s   default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 node(s) didn't match Pod's node affinity/selector.
...

# 为节点打对应标签后，Pod 马上被调度成功
root@k8s-master:~# kubectl label nodes k8s-node-1 zone=foo
node/k8s-node-1 labeled
root@k8s-master:~# kubectl get pod
NAME          READY   STATUS    RESTARTS   AGE
pod-na-demo   1/1     Running   0          119s
  • pod-nodeAffinity-preferred.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-na-demo
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - preference:
          matchExpressions:
          - key: zone
            operator: In
            values:
            - bar
        weight: 60

source repo / download raw
  • pod-nodeAffinity-required 执行
# 创建
root@k8s-master:~# kubectl apply -f pod-nodeAffinity-preferred.yaml
pod/pod-na-demo created

# 查看
root@k8s-master:~# kubectl get pod
NAME          READY   STATUS    RESTARTS   AGE
pod-na-demo   1/1     Running   0          14s
root@k8s-master:~# kubectl describe pod pod-na-demo
...
  Type    Reason     Age        From               Message
  ----    ------     ----       ----               -------
  Normal  Scheduled  23s        default-scheduler  Successfully assigned default/pod-na-demo to k8s-node-2
...
root@k8s-master:~# kubectl get nodes k8s-node-2 --show-labels
NAME         STATUS   ROLES    AGE    VERSION   LABELS
k8s-node-2   Ready    <none>   2d6h   v1.23.4   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node-2,kubernetes.io/os=linux

k8s-node-2 没有对应的标签也可以调度成功。

podAffinity 示例

  • 帮助
kubectl explain pods.spec.affinity.podAffinity

有两种配置方式：

  1. preferredDuringSchedulingIgnoredDuringExecution：软亲和性，尽量满足有相同属性 Pod 的节点，不满足也能创建成功
  2. requiredDuringSchedulingIgnoredDuringExecution：硬亲和性，必须满足有相同属性 Pod 的节点，不满足创建不成功，第一个 Pod 随机创建，后续 Pod 以第一个 Pod 为选择条件
  • pod-podAffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-pa-1
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-pa-2
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          # - key: {key: app, operator: In, values: ["hello-app"]}
          - key: app
            operator: In
            values:
            - hello-app
        topologyKey: kubernetes.io/hostname  # Pod 亲和性的判断标志

source repo / download raw
  • pod-podAffinity-required.yaml 执行
# 创建
root@k8s-master:~# kubectl apply -f pod-podAffinity-required.yaml
pod/pod-pa-1 created
pod/pod-pa-2 created

# 查看，被调度到同一个节点
root@k8s-master:~# kubectl get pod -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod-pa-1   1/1     Running   0          7s    10.244.2.10   k8s-node-2   <none>           <none>
pod-pa-2   1/1     Running   0          7s    10.244.2.11   k8s-node-2   <none>           <none>
  • preferredDuringSchedulingIgnoredDuringExecution 示例与上述类似，不在提供

podAntiAffinity 示例

podAntiAffinity 和 podAffinity 类似，提供 Pod 亲和功能

  • 帮助
kubectl explain pods.spec.affinity.podAntiAffinity

有两种配置方式：

  1. preferredDuringSchedulingIgnoredDuringExecution：软亲和性，尽量满足有相同属性 Pod 的节点反亲和
  2. requiredDuringSchedulingIgnoredDuringExecution：硬亲和性，必须满足有相同属性 Pod 的节点，不满足创建不成功，第一个 Pod 随机创建，后续 Pod 以已创建的 Pod 为选择条件反亲和
  • pod-podAntiAffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-pa-1
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-pa-2
  labels:
    app: hello-app
spec:
  containers:
  - name: hello-app
    image: gcriogooglesamples/hello-app:1.0
    ports:
    - name: http
      containerPort: 8080
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          # - key: {key: app, operator: In, values: ["hello-app"]}
          - key: app
            operator: In
            values:
            - hello-app
        topologyKey: kubernetes.io/hostname  # Pod 亲和性的判断标志
        # topologyKey: rack

source repo / download raw
  • pod-podAntiAffinity-required.yaml 执行
# 创建
root@k8s-master:~# kubectl apply -f pod-podAntiAffinity-required.yaml
pod/pod-pa-1 created
pod/pod-pa-2 created

# 查看，被调度到不同节点
root@k8s-master:~# kubectl get pod -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod-pa-1   1/1     Running   0          3s    10.244.2.12   k8s-node-2   <none>           <none>
pod-pa-2   1/1     Running   0          3s    10.244.1.4    k8s-node-1   <none>           <none>

# 可以为节点打标签演示 pod 只调度一个的情况
kubectl label nodes k8s-node-1 rack=r1-1
kubectl label nodes k8s-node-2 rack=r1-1
root@k8s-master:~# kubectl get pod
NAME       READY   STATUS    RESTARTS   AGE
pod-pa-1   1/1     Running   0          5s
pod-pa-2   0/1     Pending   0          5s
root@k8s-master:~# kubectl describe pod pod-pa-2
...
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  20s (x2 over 22s)  default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 node(s) didn't match pod anti-affinity rules.
...

Taints 调度示例

实现原理：给节点打Taints（污点，键值属性数据，只用在节点上，同类的有：标签、注解），Pod 不能容忍污点无法调度。通过在 Pod 上定义 Tolerations（容忍度，键值属性数据），决定能够容忍哪些污点。

Taints 定义在 nodes.spec 中，其中，Taints 的 node.spec.taints.effect 定义对 Pod 的排斥效果：

  1. NoSchedule：不调度。仅影响调度过程，新增污点时，对现存的 Pod 对象不产生影响
  2. NoExecute：即影响调度过程，也影响现有的 Pod 对象。新增污点时，不容忍的 Pod 对象将被驱逐
  3. PreferNoSchedule：没合适的 note 时运行 Pod
  • 帮助
kubectl explain node.spec
kubectl explain node.spec.taints
  • 系统默认的容忍度
# master 节点
root@k8s-master:~# kubectl get nodes k8s-master -o yaml
...
spec:
  ...
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
...

# controller-manager
root@k8s-master:~# kubectl -n kube-system describe pod kube-controller-manager-k8s-master
...
Tolerations:       :NoExecute op=Exists
...

# flannel pod
root@k8s-master:~# kubectl -n kube-system describe pod kube-flannel-ds-btc96
...
Tolerations:                 :NoSchedule op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
...
  • Node 打污点
# 帮助
$ kubectl taint -h
Usage:
  kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_N [options]

# 给 k8s-node-1 打 Taints：生产专用
$ kubectl taint node k8s-node-1 node-type=production:NoSchedule

# 查看 k8s-node-1 的 Taints
$ kubectl describe node k8s-node-1 | grep Taints
Taints:             node-type=production:NoSchedule

# 创建多个副本的 Pod，默认是不会往 k8s-node-1 调度

# 为 k8s-node-2 打不点，非容忍 uat 的都删除
kubectl taint node k8s-node-2 node-type=uat:NoExecute

# 删除
$ kubectl taint node k8s-node-1 node-type-
  • Pod 容忍污点
kubectl explain pods.spec.tolerations
  • pod-tolarations-taints.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-app-dp
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hello-app
      release: canary
  template:
    metadata:
      name: hello-app-pod
      labels:
        app: hello-app
        release: canary
    spec:
      containers:
      - name: hello-app-1
        image: gcriogooglesamples/hello-app:2.0
        ports:
        - name: http
          containerPort: 8080
      tolerations:
      - key: "node-type"
        operator: "Equal"  # Exists and Equal
        value: "uat"
        effect: "NoSchedule"
        # effect: "" # tolerate all taints
        # tolerationSeconds: 3600  # only work NoExecute

source repo / download raw
  • 执行
# 创建，默认 Pending 状态
root@k8s-master:~/manifests/schedule# kubectl apply -f pod-tolarations-taints.yaml
deployment.apps/hello-app-dp configured

# 为 k8s-node-2 打污点
root@k8s-master:~/manifests/schedule# kubectl taint node k8s-node-2 node-type=uat:NoSchedule
node/k8s-node-2 tainted

# Pod 调度正常
root@k8s-master:~/manifests/schedule# kubectl get pod
NAME                           READY   STATUS    RESTARTS   AGE
hello-app-dp-656564887-ccsvc   1/1     Running   0          2m55s
hello-app-dp-656564887-t7wcx   1/1     Running   0          2m55s
最近更新
相关文章
最新评论
Home Archives Categories Tags Statistics
本文总阅读量 次 本站总访问量 次 本站总访客数 人