环境信息:
RKE2 版本: rke2 v1.30.5+rke2r1
节点 CPU 架构，操作系统和版本：Linux kube-node03 6.8.0-55-generic #57-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 23:42:21 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
集群配置:1server 3agents
**问题描述:**在启动rke2-server的时候，报错
重现步骤:
- 安装 RKE2 的命令:
#主机创建目录上传4个文件
mkdir -pv /data/rke2-artifacts && cd /data/rke2-artifacts
#然后上传文件
curl -OLs https://github.com/rancher/rke2/releases/download/v1.30.5%2Brke2r1/rke2-images.linux-amd64.tar.zst
curl -OLs https://github.com/rancher/rke2/releases/download/v1.30.5%2Brke2r1/rke2.linux-amd64.tar.gz
curl -OLs https://github.com/rancher/rke2/releases/download/v1.30.5%2Brke2r1/sha256sum-amd64.txt
#复制压缩包至镜像目录
mkdir -p /var/lib/rancher/rke2/agent/images && cd /data/rke2-artifacts
cp rke2-images.linux-amd64.tar.zst /var/lib/rancher/rke2/agent/images
mkdir -p /etc/rancher/rke2/
cat >/etc/rancher/rke2/config.yaml <<EOL
token: 123456
system-default-registry: registry.cn-hangzhou.aliyuncs.com
EOL
#执行安装脚本
cd /data/rke2-artifacts
INSTALL_RKE2_TYPE=“server” INSTALL_RKE2_MIRROR=cn INSTALL_RKE2_VERTSION=v1.30.5+rke2r1 INSTALL_RKE2_ARTIFACT_PATH=/data/rke2-artifacts sh install.sh
[INFO] finding release for channel stable
[INFO] using v1.28.10-rke2r1 as release
[INFO] downloading checksums at https://rancher-mirror.rancher.cn/rke2/releases/download/v1.28.10-rke2r1/sha256sum-amd64.txt
[INFO] downloading tarball at https://rancher-mirror.rancher.cn/rke2/releases/download/v1.28.10-rke2r1/rke2.linux-amd64.tar.gz
[INFO] verifying tarball
[INFO] unpacking tarball file to /usr/local
设置为开机自启并重启
systemctl enable rke2-server.service
启动服务
systemctl start rke2-server
查看日志
journalctl -fu rke2-server.service
预期结果:
实际结果:
日志
Mar 09 15:10:19 kube-master systemd[1]: Starting rke2-server.service - Rancher Kubernetes Engine v2 (server)…
Mar 09 15:10:19 kube-master sh[32444]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=warning msg=“not running in CIS mode”
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=“Applying Pod Security Admission Configuration”
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=“Starting rke2 v1.30.5+rke2r1 (0c83bc82315cd61664880d0b52a7e070e9fbd623)”
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=“Managed etcd cluster initializing”
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=“Starting etcd for new cluster, cluster-reset=false”
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=start
Mar 09 15:10:21 kube-master rke2[32456]: time=“2025-03-09T15:10:21Z” level=info msg=“schedule, now=2025-03-09T15:10:21Z, entry=1, next=2025-03-10T00:00:00Z”
Mar 09 15:10:23 kube-master rke2[32456]: time=“2025-03-09T15:10:23Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:39228: starting”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,rke2 --authorization-mode=Node,RBAC --bind-address=0.0.0.0 --cert-dir=/var/lib/rancher/rke2/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/rke2/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/rke2/server/cred/encryption-config.json --encryption-provider-config-automatic-reload=true --etcd-cafile=/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/rke2/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/rke2/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --kubelet-certificate-authority=/var/lib/rancher/rke2/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/rke2/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/rke2/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/rke2/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.key”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Removed kube-apiserver static pod manifest”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --profiling=false --secure-port=10259”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --configure-cloud-routes=false --controllers=,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --profiling=false --root-ca-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/rke2/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --use-service-account-credentials=true"
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=,-route,-service --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Waiting for cri connection: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /run/k3s/containerd/containerd.sock: connect: connection refused"”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Server node token is available at /var/lib/rancher/rke2/server/token”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“To join server node to cluster: rke2 server -s https://10.10.10.249:9345 -t {SERVER_NODE_TOKEN}"
Mar 09 15:10:28 kube-master rke2[32456]: time="2025-03-09T15:10:28Z" level=info msg="Agent node token is available at /var/lib/rancher/rke2/server/agent-token"
Mar 09 15:10:28 kube-master rke2[32456]: time="2025-03-09T15:10:28Z" level=info msg="To join agent node to cluster: rke2 agent -s https://10.10.10.249:9345 -t {AGENT_NODE_TOKEN}”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Wrote kubeconfig /etc/rancher/rke2/rke2.yaml”
Mar 09 15:10:28 kube-master rke2[32456]: time=“2025-03-09T15:10:28Z” level=info msg=“Run: rke2 kubectl”
Mar 09 15:10:44 kube-master rke2[32456]: time=“2025-03-09T15:10:44Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:54630: runtime core not ready”
Mar 09 15:10:45 kube-master rke2[32456]: time=“2025-03-09T15:10:45Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:59084: runtime core not ready”
Mar 09 15:10:46 kube-master rke2[32456]: time=“2025-03-09T15:10:46Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:35004: runtime core not ready”
Mar 09 15:10:48 kube-master rke2[32456]: time=“2025-03-09T15:10:48Z” level=info msg=“Waiting for cri connection: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /run/k3s/containerd/containerd.sock: connect: connection refused"”
Mar 09 15:10:49 kube-master rke2[32456]: time=“2025-03-09T15:10:49Z” level=info msg=“Password verified locally for node kube-master”
Mar 09 15:10:49 kube-master rke2[32456]: time=“2025-03-09T15:10:49Z” level=info msg=“certificate CN=kube-master signed by CN=rke2-server-ca@1741531192: notBefore=2025-03-09 14:39:52 +0000 UTC notAfter=2026-03-09 15:10:49 +0000 UTC”
Mar 09 15:10:52 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:10:51.983666Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:10:52 kube-master rke2[32456]: time=“2025-03-09T15:10:52Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:10:58 kube-master rke2[32456]: time=“2025-03-09T15:10:58Z” level=info msg=“certificate CN=system:node:kube-master,O=system:nodes signed by CN=rke2-client-ca@1741531192: notBefore=2025-03-09 14:39:52 +0000 UTC notAfter=2026-03-09 15:10:58 +0000 UTC”
Mar 09 15:10:58 kube-master rke2[32456]: time=“2025-03-09T15:10:58Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:10:58 kube-master rke2[32456]: time=“2025-03-09T15:10:58Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Module overlay was already loaded”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Module nf_conntrack was already loaded”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Module br_netfilter was already loaded”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Module iptable_nat was already loaded”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Module iptable_filter was already loaded”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:50128: runtime core not ready”
Mar 09 15:11:03 kube-master rke2[32456]: time=“2025-03-09T15:11:03Z” level=info msg=“Runtime image registry.cn-hangzhou.aliyuncs.com/rancher/rke2-runtime:v1.30.5-rke2r1 bin and charts directories already exist; skipping extract”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-controller-crd.yaml to set cluster configuration values”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-validation-webhook.yaml to set cluster configuration values”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-traefik-crd.yaml to set cluster configuration values”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:48230: runtime core not ready”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-cloud-provider.yaml to set cluster configuration values”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-flannel.yaml to set cluster configuration values”
Mar 09 15:11:04 kube-master rke2[32456]: time=“2025-03-09T15:11:04Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-multus.yaml to set cluster configuration values”
Mar 09 15:11:05 kube-master rke2[32456]: time=“2025-03-09T15:11:05Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-canal.yaml to set cluster configuration values”
Mar 09 15:11:05 kube-master rke2[32456]: time=“2025-03-09T15:11:05Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-traefik.yaml to set cluster configuration values”
Mar 09 15:11:05 kube-master rke2[32456]: time=“2025-03-09T15:11:05Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-csi-driver.yaml to set cluster configuration values”
Mar 09 15:11:05 kube-master rke2[32456]: time=“2025-03-09T15:11:05Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml to set cluster configuration values”
Mar 09 15:11:06 kube-master rke2[32456]: time=“2025-03-09T15:11:06Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-controller.yaml to set cluster configuration values”
Mar 09 15:11:06 kube-master rke2[32456]: time=“2025-03-09T15:11:06Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico.yaml to set cluster configuration values”
Mar 09 15:11:06 kube-master rke2[32456]: time=“2025-03-09T15:11:06Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:51844: runtime core not ready”
Mar 09 15:11:07 kube-master rke2[32456]: time=“2025-03-09T15:11:07Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-cilium.yaml to set cluster configuration values”
Mar 09 15:11:07 kube-master rke2[32456]: time=“2025-03-09T15:11:07Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx.yaml to set cluster configuration values”
Mar 09 15:11:07 kube-master rke2[32456]: time=“2025-03-09T15:11:07Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-metrics-server.yaml to set cluster configuration values”
Mar 09 15:11:07 kube-master rke2[32456]: time=“2025-03-09T15:11:07Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-cpi.yaml to set cluster configuration values”
Mar 09 15:11:08 kube-master rke2[32456]: time=“2025-03-09T15:11:08Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-csi.yaml to set cluster configuration values”
Mar 09 15:11:08 kube-master rke2[32456]: time=“2025-03-09T15:11:08Z” level=info msg=“Waiting for cri connection: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /run/k3s/containerd/containerd.sock: connect: connection refused"”
Mar 09 15:11:08 kube-master rke2[32456]: time=“2025-03-09T15:11:08Z” level=info msg=“Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico-crd.yaml to set cluster configuration values”
Mar 09 15:11:08 kube-master rke2[32456]: time=“2025-03-09T15:11:08Z” level=info msg=“Logging containerd to /var/lib/rancher/rke2/agent/containerd/containerd.log”
Mar 09 15:11:08 kube-master rke2[32456]: time=“2025-03-09T15:11:08Z” level=info msg=“Running containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/rke2/agent/containerd”
Mar 09 15:11:09 kube-master rke2[32456]: time=“2025-03-09T15:11:09Z” level=info msg=“Waiting for containerd startup: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /run/k3s/containerd/containerd.sock: connect: connection refused"”
Mar 09 15:11:10 kube-master rke2[32456]: time=“2025-03-09T15:11:10Z” level=info msg=“containerd is now running”
Mar 09 15:11:11 kube-master rke2[32456]: time=“2025-03-09T15:11:11Z” level=info msg=“Importing images from /var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst”
Mar 09 15:11:19 kube-master rke2[32456]: time=“2025-03-09T15:11:19Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:34776: runtime core not ready”
Mar 09 15:11:22 kube-master rke2[32456]: time=“2025-03-09T15:11:22Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:46558: runtime core not ready”
Mar 09 15:11:23 kube-master rke2[32456]: time=“2025-03-09T15:11:23Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:43570: runtime core not ready”
Mar 09 15:11:27 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:11:27.04929Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:11:27 kube-master rke2[32456]: time=“2025-03-09T15:11:27Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:11:28 kube-master rke2[32456]: time=“2025-03-09T15:11:28Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:11:28 kube-master rke2[32456]: time=“2025-03-09T15:11:28Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:11:28 kube-master rke2[32456]: time=“2025-03-09T15:11:28Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:11:32 kube-master rke2[32456]: time=“2025-03-09T15:11:32Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:40382: runtime core not ready”
Mar 09 15:11:43 kube-master rke2[32456]: time=“2025-03-09T15:11:43Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:46234: runtime core not ready”
Mar 09 15:11:48 kube-master rke2[32456]: time=“2025-03-09T15:11:48Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:11:51 kube-master rke2[32456]: time=“2025-03-09T15:11:51Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:47210: runtime core not ready”
Mar 09 15:11:56 kube-master rke2[32456]: time=“2025-03-09T15:11:56Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:45876: runtime core not ready”
Mar 09 15:11:58 kube-master rke2[32456]: time=“2025-03-09T15:11:58Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:11:58 kube-master rke2[32456]: time=“2025-03-09T15:11:58Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:12:00 kube-master rke2[32456]: time=“2025-03-09T15:12:00Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:34056: runtime core not ready”
Mar 09 15:12:02 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:12:02.067212Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:12:02 kube-master rke2[32456]: time=“2025-03-09T15:12:02Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:12:08 kube-master rke2[32456]: time=“2025-03-09T15:12:08Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:37326: runtime core not ready”
Mar 09 15:12:08 kube-master rke2[32456]: time=“2025-03-09T15:12:08Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:12:18 kube-master rke2[32456]: time=“2025-03-09T15:12:18Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:42174: runtime core not ready”
Mar 09 15:12:22 kube-master rke2[32456]: time=“2025-03-09T15:12:22Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:42704: runtime core not ready”
Mar 09 15:12:26 kube-master rke2[32456]: time=“2025-03-09T15:12:26Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:59228: runtime core not ready”
Mar 09 15:12:28 kube-master rke2[32456]: time=“2025-03-09T15:12:28Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:12:28 kube-master rke2[32456]: time=“2025-03-09T15:12:28Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:12:28 kube-master rke2[32456]: time=“2025-03-09T15:12:28Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:12:37 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:12:37.0914Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:12:37 kube-master rke2[32456]: time=“2025-03-09T15:12:37Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:12:38 kube-master rke2[32456]: time=“2025-03-09T15:12:38Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:52206: runtime core not ready”
Mar 09 15:12:39 kube-master rke2[32456]: time=“2025-03-09T15:12:39Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:40040: runtime core not ready”
Mar 09 15:12:41 kube-master rke2[32456]: time=“2025-03-09T15:12:41Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:54280: runtime core not ready”
Mar 09 15:12:48 kube-master rke2[32456]: time=“2025-03-09T15:12:48Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:12:55 kube-master rke2[32456]: time=“2025-03-09T15:12:55Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:53128: runtime core not ready”
Mar 09 15:12:58 kube-master rke2[32456]: time=“2025-03-09T15:12:58Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:12:58 kube-master rke2[32456]: time=“2025-03-09T15:12:58Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:12:59 kube-master rke2[32456]: time=“2025-03-09T15:12:59Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:34716: runtime core not ready”
Mar 09 15:13:00 kube-master rke2[32456]: time=“2025-03-09T15:13:00Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:43822: runtime core not ready”
Mar 09 15:13:08 kube-master rke2[32456]: time=“2025-03-09T15:13:08Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:13:12 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:13:12.114988Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:13:12 kube-master rke2[32456]: time=“2025-03-09T15:13:12Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:13:17 kube-master rke2[32456]: time=“2025-03-09T15:13:17Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:42226: runtime core not ready”
Mar 09 15:13:18 kube-master rke2[32456]: time=“2025-03-09T15:13:18Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:40756: runtime core not ready”
Mar 09 15:13:19 kube-master rke2[32456]: time=“2025-03-09T15:13:19Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:60180: runtime core not ready”
Mar 09 15:13:28 kube-master rke2[32456]: time=“2025-03-09T15:13:28Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:13:28 kube-master rke2[32456]: time=“2025-03-09T15:13:28Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:13:28 kube-master rke2[32456]: time=“2025-03-09T15:13:28Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:13:36 kube-master rke2[32456]: time=“2025-03-09T15:13:36Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:59964: runtime core not ready”
Mar 09 15:13:36 kube-master rke2[32456]: time=“2025-03-09T15:13:36Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:39742: runtime core not ready”
Mar 09 15:13:37 kube-master rke2[32456]: time=“2025-03-09T15:13:37Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:35452: runtime core not ready”
Mar 09 15:13:47 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:13:47.165551Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:13:47 kube-master rke2[32456]: time=“2025-03-09T15:13:47Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:13:49 kube-master rke2[32456]: time=“2025-03-09T15:13:49Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:13:55 kube-master rke2[32456]: time=“2025-03-09T15:13:55Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:58368: runtime core not ready”
Mar 09 15:13:56 kube-master rke2[32456]: time=“2025-03-09T15:13:56Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:50500: runtime core not ready”
Mar 09 15:13:58 kube-master rke2[32456]: time=“2025-03-09T15:13:58Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:13:58 kube-master rke2[32456]: time=“2025-03-09T15:13:58Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:13:59 kube-master rke2[32456]: time=“2025-03-09T15:13:59Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:52432: runtime core not ready”
Mar 09 15:14:08 kube-master rke2[32456]: time=“2025-03-09T15:14:08Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:14:18 kube-master rke2[32456]: time=“2025-03-09T15:14:18Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:45792: runtime core not ready”
Mar 09 15:14:20 kube-master rke2[32456]: time=“2025-03-09T15:14:20Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:53484: runtime core not ready”
Mar 09 15:14:21 kube-master rke2[32456]: time=“2025-03-09T15:14:21Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:48656: runtime core not ready”
Mar 09 15:14:22 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:14:22.180007Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}
Mar 09 15:14:22 kube-master rke2[32456]: time=“2025-03-09T15:14:22Z” level=info msg=“Failed to test data store connection: context deadline exceeded”
Mar 09 15:14:28 kube-master rke2[32456]: time=“2025-03-09T15:14:28Z” level=info msg=“Waiting for etcd server to become available”
Mar 09 15:14:28 kube-master rke2[32456]: time=“2025-03-09T15:14:28Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:14:28 kube-master rke2[32456]: time=“2025-03-09T15:14:28Z” level=info msg=“Waiting for API server to become available”
Mar 09 15:14:43 kube-master rke2[32456]: time=“2025-03-09T15:14:42Z” level=error msg=“Sending HTTP 503 response to 10.10.10.247:40262: runtime core not ready”
Mar 09 15:14:43 kube-master rke2[32456]: time=“2025-03-09T15:14:43Z” level=error msg=“Sending HTTP 503 response to 10.10.10.238:50172: runtime core not ready”
Mar 09 15:14:45 kube-master rke2[32456]: time=“2025-03-09T15:14:45Z” level=error msg=“Sending HTTP 503 response to 10.10.10.248:47932: runtime core not ready”
Mar 09 15:14:48 kube-master rke2[32456]: time=“2025-03-09T15:14:48Z” level=info msg=“Pod for etcd not synced (pod sandbox not found), retrying”
Mar 09 15:14:57 kube-master rke2[32456]: {“level”:“warn”,“ts”:“2025-03-09T15:14:57.245511Z”,“logger”:“etcd-client”,“caller”:“v3@v3.5.13-k3s1/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc0001723c0/127.0.0.1:2379”,“attempt”:0,“error”:“rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"”}