三台etcd容器主节点一直在重启

Rancher Server 设置

  • Rancher 版本:2.6.12
  • 安装选项 (Docker install/Helm Chart):
    • 如果是 Helm Chart 安装,需要提供 Local 集群的类型(RKE1, RKE2, k3s, EKS, 等)和版本:RKE1
  • 在线或离线部署:
    离线
    下游集群信息
  • Kubernetes 版本: 1.22
  • Cluster Type (Local/Downstream): Local
    • 如果 Downstream,是什么类型的集群?(自定义/导入或为托管 等):

用户信息

  • 登录用户的角色是什么? (管理员/集群所有者/集群成员/项目所有者/项目成员/自定义):管理员
    • 如果自定义,自定义权限集:

主机操作系统:
centos7.9
问题描述:
服务器断电后,恢复快照。发现etcd主节点一直重启。目前k8s可以正常使用
重现步骤:

结果:
主节点etcd一直重启
预期结果:

截图:


其他上下文信息:

日志
{"level":"info","ts":"2025-01-21T02:00:06.472Z","caller":"embed/etcd.go:139","msg":"configuring client listeners","listen-client-urls":["https://0.0.0.0:2379"]}
{"level":"info","ts":"2025-01-21T02:00:06.472Z","caller":"embed/etcd.go:307","msg":"starting an etcd server","etcd-version":"3.5.0","git-sha":"946a5a6f2","go-version":"go1.16.3","go-os":"linux","go-arch":"amd64","max-cpu-set":8,"max-cpu-available":8,"member-initialized":false,"name":"etcd-192.168.0.211","data-dir":"/var/lib/rancher/etcd/","wal-dir":"","wal-dir-dedicated":"","member-dir":"/var/lib/rancher/etcd/member","force-new-cluster":false,"heartbeat-interval":"500ms","election-timeout":"5s","initial-election-tick-advance":true,"snapshot-count":100000,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["https://192.168.0.211:2380"],"listen-peer-urls":["https://0.0.0.0:2380"],"advertise-client-urls":["https://192.168.0.211:2379"],"listen-client-urls":["https://0.0.0.0:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"etcd-192.168.0.211=https://192.168.0.211:2380,etcd-192.168.0.212=https://192.168.0.212:2380,etcd-192.168.0.213=https://192.168.0.213:2380","initial-cluster-state":"existing","initial-cluster-token":"etcd-cluster-1","quota-size-bytes":2147483648,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","auto-compaction-mode":"periodic","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"}
{"level":"info","ts":"2025-01-21T02:00:06.473Z","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"/var/lib/rancher/etcd/member/snap/db","took":"230.533µs"}
{"level":"info","ts":"2025-01-21T02:00:06.486Z","caller":"embed/etcd.go:367","msg":"closing etcd server","name":"etcd-192.168.0.211","data-dir":"/var/lib/rancher/etcd/","advertise-peer-urls":["https://192.168.0.211:2380"],"advertise-client-urls":["https://192.168.0.211:2379"]}
{"level":"info","ts":"2025-01-21T02:00:06.486Z","caller":"embed/etcd.go:369","msg":"closed etcd server","name":"etcd-192.168.0.211","data-dir":"/var/lib/rancher/etcd/","advertise-peer-urls":["https://192.168.0.211:2380"],"advertise-client-urls":["https://192.168.0.211:2379"]}
{"level":"fatal","ts":"2025-01-21T02:00:06.486Z","caller":"etcdmain/etcd.go:203","msg":"discovery failed","error":"error validating peerURLs {ClusterID:e22b46be0fa0f092 Members:[&{ID:731c07b6350ac7ba RaftAttributes:{PeerURLs:[https://192.168.0.212:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.212 ClientURLs:[https://192.168.0.212:2379]}} &{ID:89f2dd95cfd95d8b RaftAttributes:{PeerURLs:[https://192.168.0.211:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.211 ClientURLs:[https://192.168.0.211:2379]}} &{ID:915d68034c189d24 RaftAttributes:{PeerURLs:[https://192.168.0.212:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.212 ClientURLs:[https://192.168.0.212:2379]}} &{ID:b71e48c5cf69a343 RaftAttributes:{PeerURLs:[https://192.168.0.213:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.213 ClientURLs:[https://192.168.0.213:2379]}} &{ID:f275e2577e4f002 RaftAttributes:{PeerURLs:[https://192.168.0.212:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.212 ClientURLs:[https://192.168.0.212:2379]}} &{ID:40c5803182792874 RaftAttributes:{PeerURLs:[https://192.168.0.213:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.213 ClientURLs:[https://192.168.0.213:2379]}} &{ID:4208c2a616bd9194 RaftAttributes:{PeerURLs:[https://192.168.0.211:2380] IsLearner:false} Attributes:{Name:etcd-192.168.0.211 ClientURLs:[https://192.168.0.211:2379]}}] RemovedMemberIDs:[]}: member count is unequal","stacktrace":"go.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdmain/etcd.go:203\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/etcdmain/main.go:40\nmain.main\n\t/tmp/etcd-release-3.5.0/etcd/release/etcd/server/main.go:32\nruntime.main\n\t/home/remote/sbatsche/.gvm/gos/go1.16.3/src/runtime/proc.go:225"}
[root@master-211 snap]# docker ps -a -f=name=etcd$