INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC=‘server’ K3S_DATASTORE_ENDPOINT=‘mysql://username:password@tcp(hostname:3306)/database-name’ ./install.sh
INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC=‘server’ K3S_DATASTORE_ENDPOINT=‘mysql://root:123456@tcp(192.168.130.100:13306)/k3s’ --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 644 --docker’ ./install.sh
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_DATASTORE_ENDPOINT=‘mysql://root:123456@tcp(192.168.130.100:13306)/k3s’ INSTALL_K3S_EXEC=‘server --token K1035b33f7512011700683242dcd868d3178ba8c2cb492445326567741e732aad7d::server:21259f370460f60e30cc9cb095ff3017 --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 644 --docker’ ./install.sh
https://127.0.0.1:6443/api/v1/namespaces/kube-system/serviceaccounts/metrics-server/token
规划
192.168.130.102 t2(master1)
192.168.130.103 t3(master2)
192.168.130.104 t4(worker1)
192.168.130.105 t5(worker2)
192.168.130.106 t6(worker3)
5个虚拟机，操作系统是centos7.9，规划如下
192.168.130.102 t2(server1)
192.168.130.103 t3(server2)
192.168.130.104 t4(server3)
192.168.130.105 t5(agent1)
192.168.130.106 t6(agent2)
目前在t2服务器上成功部署了server，部署命令：INSTALL_K3S_SKIP_DOWNLOAD=true K3S_DATASTORE_ENDPOINT=‘mysql://root:123456@tcp(192.168.130.100:13306)/k3s’ INSTALL_K3S_EXEC=‘server --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 644 --docker’ ./install.sh
然后在t3服务器上部署server2
环境信息:
K3s 版本:
k3s version v1.24.17+k3s1 (026bb0ec)
go version go1.20.7
节点 CPU 架构、操作系统和版本：:
Linux t2 5.4.221-1.el7.elrepo.x86_64 #1 SMP Thu Oct 27 19:21:06 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
集群配置:
3 servers, 2 agents
192.168.130.102 t2(server1)
192.168.130.103 t3(server2)
192.168.130.104 t4(server3)
192.168.130.105 t5(agent1)
192.168.130.106 t6(agent2)
问题描述:
server1
复现步骤:
- 安装 K3s 的命令:
步骤1：
在server1(192.168.130.102)服务器上成功部署了server
部署命令：INSTALL_K3S_SKIP_DOWNLOAD=true K3S_DATASTORE_ENDPOINT=‘mysql://root:123456@tcp(192.168.130.100:13306)/k3s’ INSTALL_K3S_EXEC=‘server --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 644 --docker’ ./install.sh
查看server1 部署后的token
cat /var/lib/rancher/k3s/server/node-token
K1093a30971d9fa23077d7c0bccb818365044027f989f0109c43553375a82ab40ee::server:8174dc684508bd30aea0466220bc2ee5
步骤2：在server2主机上安装server节点
执行命令：INSTALL_K3S_SKIP_DOWNLOAD=true K3S_DATASTORE_ENDPOINT=‘mysql://root:123456@tcp(192.168.130.100:13306)/k3s’ INSTALL_K3S_EXEC=‘server --token K1093a30971d9fa23077d7c0bccb818365044027f989f0109c43553375a82ab40ee::server:8174dc684508bd30aea0466220bc2ee5 --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 644 --docker --disable-etcd’ ./install.sh
预期结果:
预期server2节点安装成功，和server1组成高可用版本
实际结果:
server2中报错：msg=“starting kubernetes: preparing server: bootstrap data already found and encrypted with different token”
附加上下文/日志:
日志
Nov 2 19:27:28 t3 k3s: time="2023-11-02T19:27:28+08:00" level=info msg="Active TLS secret / (ver=) (count 10): map[listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-192.168.130.103:192.168.130.103 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-t3:t3 listener.cattle.io/fingerprint:SHA1=9238563EFF6D2418AF775594680E51AA11042D3A]"
Nov 2 19:27:28 t3 k3s: time="2023-11-02T19:27:28+08:00" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Nov 2 19:27:28 t3 systemd: k3s.service: main process exited, code=exited, status=1/FAILURE
Nov 2 19:27:28 t3 systemd: Failed to start Lightweight Kubernetes.
Nov 2 19:27:28 t3 systemd: Unit k3s.service entered failed state.
Nov 2 19:27:28 t3 systemd: k3s.service failed.
Nov 2 19:27:34 t3 systemd: k3s.service holdoff time over, scheduling restart.
Nov 2 19:27:34 t3 systemd: Stopped Lightweight Kubernetes.
Nov 2 19:27:34 t3 systemd: Ignoring invalid environment assignment 'K3S_SERVER_ARGS='--token'
Nov 2 19:27:34 t3 systemd: 'K1093a30971d9fa23077d7c0bccb818365044027f989f0109c43553375a82ab40ee::server:8174dc684508bd30aea0466220bc2ee5'
Nov 2 19:27:34 t3 systemd: '--write-kubeconfig'
Nov 2 19:27:34 t3 systemd: '/root/.kube/config'
Nov 2 19:27:34 t3 systemd: '--write-kubeconfig-mode'