Failed to get unit file state for nm-cloud-setup.service

环境信息:

K3s 版本:

k3s version v1.21.7+k3s1 (ac705709)

go version go1.16.10

节点 CPU 架构、操作系统和版本::

Linux k3s1-s01 3.10.0-1160.59.1.el7.x86_64 #1 SMP Wed Feb 23 16:47:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

集群配置:

3 servers, 3 agents

问题描述:

#######################

192.168.2.53 k3s1-s01

192.168.2.54 k3s1-s02

192.168.2.55 k3s1-s03

192.168.2.56 k3s1-a01

192.168.2.57 k3s1-a02

192.168.2.58 k3s1-a03

192.168.2.60 k3s1-lb

#######################

复现步骤:

  • 安装 K3s 的命令:

master1:192.168.2.53

curl -sLS http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_TOKEN=‘k3s1v1217’ INSTALL_K3S_EXEC=‘server --tls-san 192.168.2.60 --tls-san 192.168.2.53 --tls-san 192.168.2.54 --tls-san 192.168.2.55 --node-external-ip 192.168.2.53 --cluster-cidr 10.42.0.0/16’ INSTALL_K3S_VERSION=‘v1.21.7+k3s1’ sh -"

master2:192.168.2.54

curl -sLS http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_URL=‘https://192.168.2.53:6443’ K3S_TOKEN=‘k3s1v1217’ INSTALL_K3S_EXEC=‘server --server https://192.168.2.53:6443 --tls-san 192.168.2.60 --tls-san 192.168.2.53 --tls-san 192.168.2.54 --tls-san 192.168.2.55 --node-external-ip 192.168.2.54 --cluster-cidr 10.42.0.0/16’ INSTALL_K3S_VERSION=‘v1.21.7+k3s1’ sh -"

预期结果:

实际结果:

master1执行成功,master2执行失败。

Failed to get unit file state for nm-cloud-setup.service: No such file or directory

starting kubernetes: preparing server: https://192.168.2.53:6443/v1-k3s/server-bootstrap: 401 Unauthorized"

附加上下文/日志:

日志

[root@k3s1-s02 ~]# journalctl -xe

– Subject: Unit k3s.service has begun start-up

– Defined-By: systemd

– Support: systemd-devel Info Page

– Unit k3s.service has begun starting up.

Apr 13 16:14:24 k3s1-s02 sh[2049]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service

Apr 13 16:14:24 k3s1-s02 sh[2049]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory

Apr 13 16:14:24 k3s1-s02 k3s[2058]: time=“2022-04-13T16:14:24.666443055+08:00” level=info msg=“Starting k3s v1.21.7+k3s1 (ac705709)”

Apr 13 16:14:24 k3s1-s02 k3s[2058]: time=“2022-04-13T16:14:24.752309321+08:00” level=warning msg="Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token fr

Apr 13 16:14:24 k3s1-s02 k3s[2058]: time=“2022-04-13T16:14:24.752395543+08:00” level=info msg=“Managed etcd cluster not yet initialized”

Apr 13 16:14:24 k3s1-s02 k3s[2058]: time=“2022-04-13T16:14:24.755915358+08:00” level=fatal msg=“starting kubernetes: preparing server: https://192.168.2.53:6443/v1-k3s/server-bootstrap: 401 Unauthorized”

Apr 13 16:14:24 k3s1-s02 systemd[1]: k3s.service: main process exited, code=exited, status=1/FAILURE

Apr 13 16:14:24 k3s1-s02 systemd[1]: Failed to start Lightweight Kubernetes.

– Subject: Unit k3s.service has failed

– Defined-By: systemd

– Support: systemd-devel Info Page

– Unit k3s.service has failed.

– The result is failed.

Apr 13 16:14:24 k3s1-s02 systemd[1]: Unit k3s.service entered failed state.

Apr 13 16:14:24 k3s1-s02 systemd[1]: k3s.service failed.

Apr 13 16:14:29 k3s1-s02 systemd[1]: k3s.service holdoff time over, scheduling restart.

Apr 13 16:14:29 k3s1-s02 systemd[1]: Stopped Lightweight Kubernetes.

– Subject: Unit k3s.service has finished shutting down

– Defined-By: systemd

– Support: systemd-devel Info Page

– Unit k3s.service has finished shutting down.

Apr 13 16:14:29 k3s1-s02 systemd[1]: Starting Lightweight Kubernetes…

– Subject: Unit k3s.service has begun start-up

– Defined-By: systemd

– Support: systemd-devel Info Page

– Unit k3s.service has begun starting up.

Apr 13 16:14:29 k3s1-s02 sh[2073]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service

Apr 13 16:14:29 k3s1-s02 sh[2073]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory

Apr 13 16:14:30 k3s1-s02 k3s[2082]: time=“2022-04-13T16:14:30.469335264+08:00” level=info msg=“Starting k3s v1.21.7+k3s1 (ac705709)”

Apr 13 16:14:30 k3s1-s02 k3s[2082]: time=“2022-04-13T16:14:30.568289156+08:00” level=warning msg="Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token fr

Apr 13 16:14:30 k3s1-s02 k3s[2082]: time=“2022-04-13T16:14:30.568376177+08:00” level=info msg=“Managed etcd cluster not yet initialized”

Apr 13 16:14:30 k3s1-s02 k3s[2082]: time=“2022-04-13T16:14:30.572900129+08:00” level=fatal msg=“starting kubernetes: preparing server: https://192.168.2.53:6443/v1-k3s/server-bootstrap: 401 Unauthorized”

Apr 13 16:14:30 k3s1-s02 systemd[1]: k3s.service: main process exited, code=exited, status=1/FAILURE

Apr 13 16:14:30 k3s1-s02 systemd[1]: Failed to start Lightweight Kubernetes.

– Subject: Unit k3s.service has failed

– Defined-By: systemd

– Support: systemd-devel Info Page

上下文

你这第二个 master 肯定是失败的啊,我都没见过这么装多 master 节点的方式。

如果你想安装 K3s 高可用,有两种方式,一种是外置数据库,一种是内置 ETCD 的形式,你可以参考官网:

啊,我应该少启用集群的参数了。我再试一下。

原因找到了

master1:192.168.2.53少了一个启动参数【–cluster-init’】,master2的不变。

curl -sLS http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_TOKEN='k3s1v1217' INSTALL_K3S_EXEC='server --tls-san 192.168.2.60 --tls-san 192.168.2.53 --tls-san 192.168.2.54 --tls-san 192.168.2.55 --node-external-ip 192.168.2.53 --cluster-cidr 10.42.0.0/16 --cluster-init' INSTALL_K3S_VERSION='v1.21.7+k3s1' sh -