FAQ

发布时间: 更新时间: 总字数:684 阅读时间:2m 作者: IP上海 分享 网址

FAQ

namespace

namespace 处于 Terminating 修复

  • 方法零:k9s
  • 方法一:其他的资源删除也适用
# delete ns
kubectl patch ns/abc-ns \
    --type json \
    --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

# others
kubectl patch <some-resource>/<some-name> \
    --type json \
    --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
  • 方法二
NAMESPACE_NAME="abc-ns"
kubectl get ns $NAMESPACE_NAME  -o json > $NAMESPACE_NAME.json

修改 $NAMESPACE_NAME.json

    "metadata": {
        "finalizers": [
            "finalizers.kubesphere.io/namespaces"
        ],
        ...
    }
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },

更改为:

    "metadata": {
        "finalizers": [],
        ...
    },
    "spec": {},

API代理

# kubectl proxy --port=8081
Starting to serve on 127.0.0.1:8081
  • 删除
curl -k -H "Content-Type:application/json" -X PUT --data-binary @$NAMESPACE_NAME.json http://127.0.0.1:8081/api/v1/namespaces/$NAMESPACE_NAME/finalize

kubelet

Failed to get system container stats

  • 错误日志
Mar 12 09:37:12 [localhost] kubelet[1725]: E0312 09:37:12.570955    1725 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
  • 解决方法

编辑/etc/systemd/system/kubelet.service文件,添加:

[Service]
CPUAccounting=true
MemoryAccounting=true
  • 执行
systemctl daemon-reload
systemctl restart kubelet

spec.selector: Invalid value: v1.LabelSelector field is immutable

API apps/v1Deployment 标签选择器创建后,不可变,删除重建资源即可。

Pod

Pod 无法通过 Service IP 连接到它本身

如果 Service 的末端尝试访问自己的 Service VIP,则该端点可以把流量负载均衡回来到它们自身。 hairpin-mode(发夹模式) 标志必须被设置为 hairpin-veth 或者 promiscuous-bridge。

若为 hairpin-veth 模式,通过如下方式检测:

$ cat /sys/devices/virtual/net/cni0/brif/veth80c6661f/hairpin_mode
1

若为 promiscuous-bridge 模式,通过如下方式检测:

ip a show cni0 | grep PROMISC

orphaned pod pod_id found

孤儿 pod 删除残留问题,错误日志

orphaned pod pod_id found, but error not a directory occurred when trying to remove the volumes dir
  • 解决方法
cd /var/lib/kubelet/pods/${pod_id}/volumes/kubernetes.io~csi/pvc-${pvc_id}/
rm vol_data.json

Pod 一直处于 terminating

# 方法一
kubectl -n [namespace] delete pod [pod name] --force --grace-period=0

# 方法二:etcd 删除
export ETCDCTL_API=3
etcdctl del /registry/pods/<default>/<pod-to-be-deleted-0>

# 方法三
kubectl get pod <pod_name> -n <namespace> -o yaml > pod.yaml
# 删除 yaml 中的如下内容
metadata:
  ...
  finalizers:
  - kubernetes

kubectl delete pod <pod_name> -n <namespace>
kubectl apply -f pod.yaml

# 方法四
kubectl patch pod <pod_name> -n <namespace> -p '{"metadata":{"finalizers":null}}'

volume not registered

k8s 错误事件 event

MountVolume.SetUp failed for volume ... not registered

原因分析:

  1. 调用 kube-apiserver 异常导致,需要排查 kube-apiserver 服务的日志等、kube-apiserver-xxx 重启
  2. 可能因为 kube-apiserver 被限速,可自动通过重试恢复,参数如下:
--kube-api-burst int32   Burst to use while talking with kubernetes apiserver. Doesn't cover events and node heartbeat apis which rate limiting is   controlled by a different set of flags (default 10)
--kube-api-qps int32     QPS to use while talking with kubernetes apiserver. Doesn't cover events and node heartbeat apis which rate limiting is controlled by a different set of flags (default 5)

参考

  1. https://learnk8s.io/troubleshooting-deployments
Home Archives Categories Tags Statistics
本文总阅读量 次 本站总访问量 次 本站总访客数