环境信息:
Rancher Server 设置
Rancher 版本：2.7.9
安装选项 (Docker install/Helm Chart): Helm Chart
k3s版本：v1.26.9+k3s1
在线或离线部署：离线部署
**主机操作系统：三台ubtuntu22.04.6，两台server，一台agent，都已关闭防火墙
问题描述:
Rancher 无法访问
附加上下文/日志:
日志
**查看pods运行情况**
kubectl get pods -n kube-system
E0526 09:09:41.645390 340032 memcache.go:287] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:09:41.659942 340032 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:09:41.661778 340032 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:09:41.664870 340032 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
NAME READY STATUS RESTARTS AGE
svclb-jdchain-blance-228aaa0b-6l6gs 1/1 Running 2 (2d22h ago) 424d
svclb-anythingllm-loadbalancer-727b8af7-2wb8q 1/1 Running 2 (2d22h ago) 97d
traefik-57c84cf78d-cc6br 0/1 ContainerCreating 1 2d19h
local-path-provisioner-76d776f6f9-bkbjc 1/1 Running 0 2d8h
svclb-ollama-loadbalancer-4863a12a-gp5km 1/1 Running 0 99d
svclb-dify-bl-2451c7fe-d7752 1/1 Running 0 83d
svclb-pgsql-bl-5a504e6f-8vmmn 1/1 Running 0 83d
svclb-dify-api-bl-787d9c8c-6pths 1/1 Running 0 74d
svclb-dify-weaviate-bl-fbd122d6-t2jxh 1/1 Running 0 10d
svclb-liangsyh-loadbalancer-a7dfcd05-j2qbz 1/1 Running 0 485d
svclb-traefik-ca7181d8-b624f 2/2 Running 0 503d
svclb-activemq-loadbalancer-14fb1ed9-5rvcr 2/2 Running 0 500d
svclb-dify-weaviate-bl-fbd122d6-2km7n 1/1 Running 0 2d19h
svclb-anythingllm-loadbalancer-727b8af7-5hb74 1/1 Running 0 2d19h
svclb-traefik-ca7181d8-5v2p5 2/2 Running 0 2d19h
svclb-dify-bl-2451c7fe-g856l 1/1 Running 0 2d19h
svclb-dify-api-bl-787d9c8c-n7252 1/1 Running 0 2d19h
svclb-jdchain-blance-228aaa0b-74trl 1/1 Running 0 2d19h
svclb-activemq-loadbalancer-14fb1ed9-v49bs 2/2 Running 0 2d19h
svclb-ollama-loadbalancer-4863a12a-2tpxm 1/1 Running 0 2d19h
svclb-liangsyh-loadbalancer-a7dfcd05-grcgk 1/1 Running 0 2d19h
svclb-pgsql-bl-5a504e6f-pfnsp 1/1 Running 0 2d19h
coredns-59b4f5bbd5-bgp48 1/1 Running 0 2d18h
metrics-server-68cf49699b-4ff67 1/1 Running 0 61m
helm-install-traefik-p56rq 0/1 CrashLoopBackOff 680 (4m49s ago) 2d17h
helm-install-traefik-crd-p9r6c 0/1 CrashLoopBackOff 677 (2m20s ago) 2d17h
**查看pod运行情况**
kubectl get pods -n cattle-system
E0526 09:38:21.699386 340344 memcache.go:287] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:38:26.701568 340344 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:38:31.705988 340344 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0526 09:38:36.709516 340344 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
NAME READY STATUS RESTARTS AGE
rancher-6b6b974475-wbn9s 0/1 Error 19 2d21h
rancher-6b6b974475-txjcx 0/1 Error 15 2d19h
rancher-6b6b974475-khq6f 0/1 ContainerCreating 15 (2d19h ago) 2d19h
helm-operation-l8nf8 0/2 Error 1 2d19h
rancher-webhook-7dc5857799-lpsrp
**查看metrics-server-68cf49699b-4ff67 配置**
**k3s日志**
webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:15:46 k3s-master k3s[339476]: I0526 09:15:46.043446 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:15:46 k3s-master k3s[339476]: time="2025-05-26T09:15:46+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:46 k3s-master kernel: [242917.195132] nfs: server 10.43.152.27 not responding, timed out
May 26 09:15:47 k3s-master k3s[339476]: time="2025-05-26T09:15:47+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:48 k3s-master k3s[339476]: time="2025-05-26T09:15:48+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:49 k3s-master k3s[339476]: time="2025-05-26T09:15:49+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:50 k3s-master k3s[339476]: time="2025-05-26T09:15:50+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:50 k3s-master k3s[339476]: E0526 09:15:50.885567 339476 available_controller.go:456] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1: Get "https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 26 09:15:50 k3s-master k3s[339476]: I0526 09:15:50.932502 339476 trace.go:236] Trace[799792701]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:40.556) (total time: 130375ms):
May 26 09:15:50 k3s-master k3s[339476]: Trace[799792701]: [2m10.375578839s] [2m10.375578839s] END
May 26 09:15:50 k3s-master k3s[339476]: I0526 09:15:50.932580 339476 trace.go:236] Trace[1669590953]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:40.556) (total time: 130376ms):
May 26 09:15:50 k3s-master k3s[339476]: Trace[1669590953]: [2m10.376027304s] [2m10.376027304s] END
May 26 09:15:50 k3s-master k3s[339476]: I0526 09:15:50.932870 339476 trace.go:236] Trace[857053381]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:40.556) (total time: 130376ms):
May 26 09:15:50 k3s-master k3s[339476]: Trace[857053381]: [2m10.376346863s] [2m10.376346863s] END
May 26 09:15:50 k3s-master k3s[339476]: I0526 09:15:50.933066 339476 trace.go:236] Trace[1068626721]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:40.556) (total time: 130376ms):
May 26 09:15:50 k3s-master k3s[339476]: Trace[1068626721]: [2m10.376527304s] [2m10.376527304s] END
May 26 09:15:50 k3s-master k3s[339476]: I0526 09:15:50.933313 339476 trace.go:236] Trace[402534906]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:40.556) (total time: 130376ms):
May 26 09:15:50 k3s-master k3s[339476]: Trace[402534906]: [2m10.376484982s] [2m10.376484982s] END
May 26 09:15:51 k3s-master k3s[339476]: W0526 09:15:51.031974 339476 dispatcher.go:204] Failed calling webhook, failing closed rancher.cattle.io.secrets: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:15:51 k3s-master k3s[339476]: I0526 09:15:51.032877 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:15:51 k3s-master k3s[339476]: time="2025-05-26T09:15:51+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:52 k3s-master k3s[339476]: time="2025-05-26T09:15:52+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:53 k3s-master k3s[339476]: time="2025-05-26T09:15:53+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:54 k3s-master k3s[339476]: time="2025-05-26T09:15:54+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:55 k3s-master k3s[339476]: I0526 09:15:55.030204 339476 trace.go:236] Trace[125564393]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:45.557) (total time: 129472ms):
May 26 09:15:55 k3s-master k3s[339476]: Trace[125564393]: [2m9.472497374s] [2m9.472497374s] END
May 26 09:15:55 k3s-master k3s[339476]: I0526 09:15:55.030912 339476 trace.go:236] Trace[670599592]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:45.557) (total time: 129473ms):
May 26 09:15:55 k3s-master k3s[339476]: Trace[670599592]: [2m9.473164003s] [2m9.473164003s] END
May 26 09:15:55 k3s-master k3s[339476]: I0526 09:15:55.030913 339476 trace.go:236] Trace[1614112011]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:45.557) (total time: 129473ms):
May 26 09:15:55 k3s-master k3s[339476]: Trace[1614112011]: [2m9.473314051s] [2m9.473314051s] END
May 26 09:15:55 k3s-master k3s[339476]: I0526 09:15:55.031634 339476 trace.go:236] Trace[925978820]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:45.557) (total time: 129473ms):
May 26 09:15:55 k3s-master k3s[339476]: Trace[925978820]: [2m9.473780137s] [2m9.473780137s] END
May 26 09:15:55 k3s-master k3s[339476]: I0526 09:15:55.032114 339476 trace.go:236] Trace[69834777]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:45.557) (total time: 129474ms):
May 26 09:15:55 k3s-master k3s[339476]: Trace[69834777]: [2m9.474213985s] [2m9.474213985s] END
May 26 09:15:55 k3s-master k3s[339476]: time="2025-05-26T09:15:55+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:55 k3s-master kernel: [242926.154788] nfs: server 10.43.152.27 not responding, timed out
May 26 09:15:55 k3s-master k3s[339476]: E0526 09:15:55.890562 339476 available_controller.go:456] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1: Get "https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 26 09:15:56 k3s-master k3s[339476]: W0526 09:15:55.999998 339476 dispatcher.go:204] Failed calling webhook, failing closed rancher.cattle.io.secrets: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:15:56 k3s-master k3s[339476]: I0526 09:15:56.000509 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:15:56 k3s-master k3s[339476]: time="2025-05-26T09:15:56+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:57 k3s-master k3s[339476]: time="2025-05-26T09:15:57+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:58 k3s-master k3s[339476]: time="2025-05-26T09:15:58+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:15:59 k3s-master k3s[339476]: time="2025-05-26T09:15:59+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:00 k3s-master k3s[339476]: time="2025-05-26T09:16:00+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:00 k3s-master k3s[339476]: E0526 09:16:00.899846 339476 available_controller.go:456] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1: Get "https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
May 26 09:16:00 k3s-master k3s[339476]: W0526 09:16:00.998960 339476 dispatcher.go:204] Failed calling webhook, failing closed rancher.cattle.io.secrets: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:16:00 k3s-master k3s[339476]: I0526 09:16:00.999417 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:16:01 k3s-master k3s[339476]: I0526 09:16:01.172713 339476 trace.go:236] Trace[810147232]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:50.572) (total time: 130600ms):
May 26 09:16:01 k3s-master k3s[339476]: Trace[810147232]: [2m10.600071481s] [2m10.600071481s] END
May 26 09:16:01 k3s-master k3s[339476]: I0526 09:16:01.172714 339476 trace.go:236] Trace[1129002975]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:50.572) (total time: 130600ms):
May 26 09:16:01 k3s-master k3s[339476]: Trace[1129002975]: [2m10.600043694s] [2m10.600043694s] END
May 26 09:16:01 k3s-master k3s[339476]: I0526 09:16:01.172799 339476 trace.go:236] Trace[2006425504]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:50.572) (total time: 130600ms):
May 26 09:16:01 k3s-master k3s[339476]: Trace[2006425504]: [2m10.600152504s] [2m10.600152504s] END
May 26 09:16:01 k3s-master k3s[339476]: I0526 09:16:01.172880 339476 trace.go:236] Trace[913528700]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:50.572) (total time: 130600ms):
May 26 09:16:01 k3s-master k3s[339476]: Trace[913528700]: [2m10.60008374s] [2m10.60008374s] END
May 26 09:16:01 k3s-master k3s[339476]: I0526 09:16:01.173095 339476 trace.go:236] Trace[1703270175]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:50.572) (total time: 130600ms):
May 26 09:16:01 k3s-master k3s[339476]: Trace[1703270175]: [2m10.600300707s] [2m10.600300707s] END
May 26 09:16:01 k3s-master k3s[339476]: time="2025-05-26T09:16:01+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:02 k3s-master k3s[339476]: E0526 09:16:02.592590 339476 controller.go:113] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: Error, could not get list of group versions for APIService
May 26 09:16:02 k3s-master k3s[339476]: I0526 09:16:02.592619 339476 controller.go:126] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
May 26 09:16:02 k3s-master k3s[339476]: E0526 09:16:02.593186 339476 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: request timed out, Header: map[]
May 26 09:16:02 k3s-master k3s[339476]: I0526 09:16:02.593659 339476 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
May 26 09:16:02 k3s-master k3s[339476]: time="2025-05-26T09:16:02+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:03 k3s-master k3s[339476]: I0526 09:16:03.220483 339476 trace.go:236] Trace[205072606]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:52.437) (total time: 130782ms):
May 26 09:16:03 k3s-master k3s[339476]: Trace[205072606]: [2m10.782558039s] [2m10.782558039s] END
May 26 09:16:03 k3s-master k3s[339476]: time="2025-05-26T09:16:03+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:04 k3s-master k3s[339476]: time="2025-05-26T09:16:04+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:05 k3s-master k3s[339476]: I0526 09:16:05.268494 339476 trace.go:236] Trace[1174083779]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:55.573) (total time: 129695ms):
May 26 09:16:05 k3s-master k3s[339476]: Trace[1174083779]: [2m9.695219514s] [2m9.695219514s] END
May 26 09:16:05 k3s-master k3s[339476]: I0526 09:16:05.268494 339476 trace.go:236] Trace[1430380764]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:55.573) (total time: 129695ms):
May 26 09:16:05 k3s-master k3s[339476]: Trace[1430380764]: [2m9.695315913s] [2m9.695315913s] END
May 26 09:16:05 k3s-master k3s[339476]: I0526 09:16:05.268494 339476 trace.go:236] Trace[1392557590]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:55.573) (total time: 129695ms):
May 26 09:16:05 k3s-master k3s[339476]: Trace[1392557590]: [2m9.695409489s] [2m9.695409489s] END
May 26 09:16:05 k3s-master k3s[339476]: I0526 09:16:05.268666 339476 trace.go:236] Trace[1043470432]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:55.573) (total time: 129695ms):
May 26 09:16:05 k3s-master k3s[339476]: Trace[1043470432]: [2m9.695650993s] [2m9.695650993s] END
May 26 09:16:05 k3s-master k3s[339476]: I0526 09:16:05.268674 339476 trace.go:236] Trace[985886152]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:13:55.573) (total time: 129695ms):
May 26 09:16:05 k3s-master k3s[339476]: Trace[985886152]: [2m9.695586254s] [2m9.695586254s] END
May 26 09:16:05 k3s-master k3s[339476]: time="2025-05-26T09:16:05+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:05 k3s-master k3s[339476]: E0526 09:16:05.908519 339476 available_controller.go:456] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
May 26 09:16:05 k3s-master k3s[339476]: W0526 09:16:05.999833 339476 dispatcher.go:204] Failed calling webhook, failing closed rancher.cattle.io.secrets: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:16:06 k3s-master k3s[339476]: I0526 09:16:06.000627 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:16:06 k3s-master kernel: [242936.650538] nfs: server 10.43.152.27 not responding, timed out
May 26 09:16:06 k3s-master k3s[339476]: time="2025-05-26T09:16:06+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:07 k3s-master k3s[339476]: time="2025-05-26T09:16:07+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:08 k3s-master k3s[339476]: time="2025-05-26T09:16:08+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:09 k3s-master k3s[339476]: time="2025-05-26T09:16:09+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:10 k3s-master k3s[339476]: time="2025-05-26T09:16:10+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
May 26 09:16:10 k3s-master k3s[339476]: E0526 09:16:10.922372 339476 available_controller.go:456] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1: Get "https://10.42.1.170:10250/apis/metrics.k8s.io/v1beta1": context deadline exceeded
May 26 09:16:11 k3s-master k3s[339476]: W0526 09:16:11.002940 339476 dispatcher.go:204] Failed calling webhook, failing closed rancher.cattle.io.secrets: failed calling webhook "rancher.cattle.io.secrets": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s": no endpoints available for service "rancher-webhook"
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.003683 339476 event.go:294] "Event occurred" object="k3s-master" fieldPath="" kind="Node" apiVersion="" type="Warning" reason="NodePasswordValidationFailed" message="Deferred node password secret validation failed: Internal error occurred: failed calling webhook \"rancher.cattle.io.secrets\": failed to call webhook: Post \"https://rancher-webhook.cattle-system.svc:443/v1/webhook/mutation/secrets?timeout=15s\": no endpoints available for service \"rancher-webhook\""
May 26 09:16:11 k3s-master k3s[339476]: W0526 09:16:11.289723 339476 garbagecollector.go:752] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412618 339476 trace.go:236] Trace[720014838]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.095) (total time: 130316ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[720014838]: [2m10.316753496s] [2m10.316753496s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412689 339476 trace.go:236] Trace[1374902485]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.093) (total time: 130318ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[1374902485]: [2m10.318899026s] [2m10.318899026s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412743 339476 trace.go:236] Trace[1800124716]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.095) (total time: 130317ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[1800124716]: [2m10.317486738s] [2m10.317486738s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412796 339476 trace.go:236] Trace[1810129219]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.590) (total time: 129822ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[1810129219]: [2m9.822236709s] [2m9.822236709s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412886 339476 trace.go:236] Trace[1538231606]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:00.586) (total time: 130826ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[1538231606]: [2m10.826033877s] [2m10.826033877s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.412967 339476 trace.go:236] Trace[632524332]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.202) (total time: 130210ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[632524332]: [2m10.210458243s] [2m10.210458243s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413052 339476 trace.go:236] Trace[329317791]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:00.586) (total time: 130826ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[329317791]: [2m10.826179463s] [2m10.826179463s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413086 339476 trace.go:236] Trace[1516428921]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:00.586) (total time: 130826ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[1516428921]: [2m10.82610492s] [2m10.82610492s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413128 339476 trace.go:236] Trace[642445179]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:00.586) (total time: 130826ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[642445179]: [2m10.826181738s] [2m10.826181738s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413208 339476 trace.go:236] Trace[38365685]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:00.586) (total time: 130826ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[38365685]: [2m10.826600969s] [2m10.826600969s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413257 339476 trace.go:236] Trace[638874405]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.114) (total time: 130298ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[638874405]: [2m10.298309375s] [2m10.298309375s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413279 339476 trace.go:236] Trace[652034886]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.101) (total time: 130312ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[652034886]: [2m10.312019742s] [2m10.312019742s] END
May 26 09:16:11 k3s-master k3s[339476]: I0526 09:16:11.413399 339476 trace.go:236] Trace[432243606]: "Proxy via http_connect protocol over tcp" address:10.42.1.170:10250 (26-May-2025 09:14:01.099) (total time: 130313ms):
May 26 09:16:11 k3s-master k3s[339476]: Trace[432243606]: [2m10.313699843s] [2m10.313699843s] END
May 26 09:16:11 k3s-master k3s[339476]: time="2025-05-26T09:16:11+08:00" level=info msg="Waiting for control-plane node k3s-master startup: nodes \"k3s-master\" not found"
如果您在生产环境中使用 Rancher，希望获得更专业、及时的技术支持，也欢迎了解一下我们的商业订阅服务。可以点击论坛右上角聊天（
）图标，私信联系我了解详细信息，我们有中文支持团队为您服务 