FAQ-谢先斌的博客

FAQ

发布时间： 2020-10-07 更新时间： 2023-12-03 总字数：588 阅读时间：2m 作者： 谢先斌

FAQ

namespace

namespace 处于 Terminating 修复

方法一：其他的资源删除也适用

# delete ns
kubectl patch ns/abc-ns \
    --type json \
    --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

# others
kubectl patch <some-resource>/<some-name> \
    --type json \
    --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

方法二

NAMESPACE_NAME="abc-ns"
kubectl get ns $NAMESPACE_NAME  -o json > $NAMESPACE_NAME.json

修改 $NAMESPACE_NAME.json

    "metadata": {
        "finalizers": [
            "finalizers.kubesphere.io/namespaces"
        ],
        ...
    }
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },

更改为：

    "metadata": {
        "finalizers": [],
        ...
    },
    "spec": {},

API代理

# kubectl proxy --port=8081
Starting to serve on 127.0.0.1:8081

删除

curl -k -H "Content-Type:application/json" -X PUT --data-binary @$NAMESPACE_NAME.json http://127.0.0.1:8081/api/v1/namespaces/$NAMESPACE_NAME/finalize

kubelet

Failed to get system container stats

错误日志

Mar 12 09:37:12 [localhost] kubelet[1725]: E0312 09:37:12.570955    1725 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"

解决方法

编辑/etc/systemd/system/kubelet.service文件，添加：

[Service]
CPUAccounting=true
MemoryAccounting=true

执行

systemctl daemon-reload
systemctl restart kubelet

spec.selector: Invalid value: v1.LabelSelector field is immutable

API apps/v1 中 Deployment 标签选择器创建后，不可变，删除重建资源即可。

Pod 无法通过 Service IP 连接到它本身

如果 Service 的末端尝试访问自己的 Service VIP，则该端点可以把流量负载均衡回来到它们自身。 hairpin-mode（发夹模式）标志必须被设置为 hairpin-veth 或者 promiscuous-bridge。

若为 hairpin-veth 模式，通过如下方式检测：

$ cat /sys/devices/virtual/net/cni0/brif/veth80c6661f/hairpin_mode
1

若为 promiscuous-bridge 模式，通过如下方式检测：

ip a show cni0 | grep PROMISC

orphaned pod pod_id found

孤儿 pod 删除残留问题，错误日志

orphaned pod pod_id found, but error not a directory occurred when trying to remove the volumes dir

解决方法

cd /var/lib/kubelet/pods/${pod_id}/volumes/kubernetes.io~csi/pvc-${pvc_id}/
rm vol_data.json

volume not registered

k8s 错误事件 event

MountVolume.SetUp failed for volume ... not registered

原因分析：

调用 kube-apiserver 异常导致，需要排查 kube-apiserver 服务的日志等、kube-apiserver-xxx 重启
可能因为 kube-apiserver 被限速，可自动通过重试恢复，参数如下：

--kube-api-burst int32   Burst to use while talking with kubernetes apiserver. Doesn't cover events and node heartbeat apis which rate limiting is   controlled by a different set of flags (default 10)
--kube-api-qps int32     QPS to use while talking with kubernetes apiserver. Doesn't cover events and node heartbeat apis which rate limiting is controlled by a different set of flags (default 5)

参考

FAQ