k8s Leaderelection 集群领导选举开发

发布时间: 更新时间: 总字数:748 阅读时间:2m 作者: IP上海 分享 网址

在Kubernetes集群中,领导者选举(Leader Election)可用于确保在任何给定时间内只有一个实例负责执行领导者特定的任务,本质上是一个分布式资源锁

介绍

  • 在分布式系统中,通过服务冗余来减少单点故障

  • 在 Kubernetes 中,kube-schdulerkube-controller-manager 都是通过部署多副本来保证高可用,实际只有一个示例在运行

  • k8s 通过 client-goleaderelection 的选主机制,保证只有获取到 leader 锁的处于工作状态,且当 leader 因故故障后,其他节点竞争称为新的 leader 保证组件正常工作

  • 原理

    • 通过 etcd 实现锁,支持的对象是Lease、ConfigMap或Endpoint
    • 若领导者未能在给定的间隔内更新时间戳,会被认为是该节点已经失效,其他节点重新竞争领导者
  • 支持

    • ResourceLockConfig
    • LeaderElectionRecord
  • 启动 kubernetes 集群安装部署

代码

leaderelection resourcelock ...
package main

import (
	"context"
	"flag"
	"log"
	"os"
	"os/signal"
	"path/filepath"
	"syscall"
	"time"

	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
	"k8s.io/client-go/tools/leaderelection"
	"k8s.io/client-go/tools/leaderelection/resourcelock"
	"k8s.io/client-go/util/homedir"
	"k8s.io/klog"
)

func main() {
	klog.InitFlags(nil)

	var (
		name       string                  // 节点名称
		lockKey    string                  //锁名称
		lockNS     string                  // 锁在k8s的命名空间
		podName    = os.Getenv("POD_NAME") // Pod 名称
		kubeconfig *string
	)

	flag.StringVar(&name, "name", podName, "identity name who holder the lock")
	flag.StringVar(&lockKey, "key", "example", "lease lock name")
	flag.StringVar(&lockNS, "ns", "default", "which namespace the lease lock create in")

	// https://github.com/kubernetes/client-go/blob/release-1.30/examples/out-of-cluster-client-configuration/main.go
	if home := homedir.HomeDir(); home != "" {
		kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
	} else {
		kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
	}
	flag.Parse()

	// config := clientcmd.GetConfigFromFileOrDie(*kubeconfig)
	config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
	if err != nil {
		panic(err.Error())
	}

	// create the clientset
	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		panic(err.Error())
	}

	// Canceling this context releases resources associated with it, so code should
	// call cancel as soon as the operations running in this Context complete.
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// A Signal represents an operating system signal.
	ch := make(chan os.Signal, 1)
	signal.Notify(ch, os.Interrupt, syscall.SIGTERM)
	go func() {
		<-ch
		log.Printf("Received termination, shutdown ...")
		cancel()
	}()

	// start the leader election code loop
	leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
		Lock: &resourcelock.LeaseLock{
			LeaseMeta: metav1.ObjectMeta{
				Name:      lockKey,
				Namespace: lockNS,
			},
			Client: clientset.CoordinationV1(),
			LockConfig: resourcelock.ResourceLockConfig{
				Identity: name,
			},
		},
		ReleaseOnCancel: true,
		LeaseDuration:   60 * time.Second,
		RenewDeadline:   15 * time.Second,
		RetryPeriod:     5 * time.Second,
		Callbacks: leaderelection.LeaderCallbacks{
			OnStartedLeading: func(ctx context.Context) {
				klog.Infof("%s: get leading", name)
			},
			OnStoppedLeading: func() {
				klog.Infof("%s: lost leading", name)
			},
			OnNewLeader: func(identity string) {
				if identity == name {
					// I just got the lock
					return
				}
				klog.Infof("new leader elected: %v", identity)
			},
		},
	})

	log.Printf("%s: exit", name)
}

以上示例基于 k8s.io/client-go v0.29.3 实现

  • help
help ...
$ go run resourcelock.go -h
...
  -key string
        lease lock name (default "example")
  -kubeconfig string
        (optional) absolute path to the kubeconfig file (default "/Users/xiexianbin/.kube/config")
...
  -name string
        identity name who holder the lock
  -ns string
        which namespace the lease lock create in (default "default")
...
  • run
# lock1
$ go run resourcelock.go -ns default -name lock1
.. leaderelection.go:250] attempting to acquire leader lease default/example...
.. leaderelection.go:260] successfully acquired lease default/example
.. resourcelock.go:89] lock1: get leading
^C.. Received termination, shutdown ...
.. resourcelock.go:92] lock1: lost leading
.. lock1: exit

# lock2
$ go run resourcelock.go -ns default -name lock2
.. leaderelection.go:250] attempting to acquire leader lease default/example...
.. resourcelock.go:99] new leader elected: lock1
.. leaderelection.go:260] successfully acquired lease default/example
.. resourcelock.go:89] lock2: get leading

参考

  1. https://pkg.go.dev/k8s.io/client-go@v0.29.3/tools/leaderelection/resourcelock
  2. https://mjasion.pl/posts/golang/implementing-leader-election-in-go-using-kubernetes-api/
Home Archives Categories Tags Statistics
本文总阅读量 次 本站总访问量 次 本站总访客数