环境信息:
RKE2 版本: 1.31.14
节点 CPU 架构，操作系统和版本：
amd64 unbuntu20.04
集群配置:
1个master
问题描述:
我们也遇到了类似的问题，请问下是否有进展？
@nachonana @ksd 麻烦帮忙分析下
之前在已有harvester集群添加节点没问题，最近不能添加节点了
相关版本信息：
harvester 1.4.2
rancher-2.10.1-dirty.commit-47da90f
rke2 v1.31.4+rke2r1
在添加的节点查看rancherd日志
journalctl -u rancherd -f
Jul 22 03:58:30 hci-**-** rancherd[2955]: time="2026-07-22T03:58:30Z" level=info msg="[stdout]: [INFO] systemd: Creating service file"
Jul 22 03:58:30 hci-**-** rancherd[2955]: time="2026-07-22T03:58:30Z" level=info msg="[stdout]: [INFO] Creating environment file /etc/systemd/system/rancher-system-agent.env"
Jul 22 03:58:30 hci-**-** rancherd[2955]: time="2026-07-22T03:58:30Z" level=info msg="[stdout]: [INFO] /usr/local is unsuitable for installation: adding fallback path to systemd unit env file."
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="[stdout]: [INFO] Enabling rancher-system-agent.service"
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="[stderr]: Created symlink /etc/systemd/system/multi-user.target.wants/rancher-system-agent.service → /etc/systemd/system/rancher-system-agent.service."
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="[stdout]: [INFO] Starting/restarting rancher-system-agent.service"
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="No image provided, creating empty working directory /var/lib/rancher/rancherd/plan/work/20260722-035829-applied.plan/_2"
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="Running command: /usr/bin/rancherd [probe]"
Jul 22 03:58:31 hci-**-** rancherd[2955]: time="2026-07-22T03:58:31Z" level=info msg="[stderr]: time=\"2026-07-22T03:58:31Z\" level=info msg=\"Running probes defined in /var/lib/rancher/rancherd/plan/plan.json\""
Jul 22 03:58:32 hci-**-** rancherd[2955]: time="2026-07-22T03:58:32Z" level=info msg="[stderr]: time=\"2026-07-22T03:58:32Z\" level=info msg=\"Probe [kubelet] is unhealthy\""
rancher-system-agent.service日志
Jul 22 09:30:35 hci-**-** rancher-system-agent[3155]: W0722 09:30:35.308171 3155 reflector.go:462] pkg/mod/github.com/rancher/client-go@v1.29.3-rancher1/tools/cache/reflector.go:229: watch of *v1.Secret ended with: an error on the server ("unable to decode an event from the watch stream: stream error: stream ID 65; INTERNAL_ERROR; received from peer") has prevented the request from succeeding
Jul 22 09:31:48 hci-**-** rancher-system-agent[3155]: W0722 09:31:48.153805 3155 reflector.go:462] pkg/mod/github.com/rancher/client-go@v1.29.3-rancher1/tools/cache/reflector.go:229: watch of *v1.Secret ended with: an error on the server ("unable to decode an event from the watch stream: stream error: stream ID 69; INTERNAL_ERROR; received from peer") has prevented the request from succeeding
Jul 22 09:34:31 hci-**-** rancher-system-agent[3155]: W0722 09:34:31.043918 3155 reflector.go:462] pkg/mod/github.com/rancher/client-go@v1.29.3-rancher1/tools/cache/reflector.go:229: watch of *v1.Secret ended with: an error on the server ("unable to decode an event from the watch stream: stream error: stream ID 73; INTERNAL_ERROR; received from peer") has prevented the request from succeeding
Jul 22 09:36:32 hci-**-** rancher-system-agent[3155]: W0722 09:36:32.151503 3155 reflector.go:462] pkg/mod/github.com/rancher/client-go@v1.29.3-rancher1/tools/cache/reflector.go:229: watch of *v1.Secret ended with: an error on the server ("unable to decode an event from the watch stream: stream error: stream ID 77; INTERNAL_ERROR; received from peer") has prevented the request from succeeding
主节点查看rke2-ingress-nginx pod日志：
kubectl -n kube-system logs -f --tail 200 -l app.kubernetes.io/name=rke2-ingress-nginx --max-log-requests 20 | grep <node-ip>
<node-ip> - - [22/Jul/2026:09:42:10 +0000] "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833796250&timeout=9m0s&timeoutSeconds=540&watch=true HTTP/2.0" 200 0 "-" "rancher-system-agent/v0.0.0 (linux/amd64) kubernetes/$Format" 148 60.001 [cattle-system-rancher-80] [] 10.52.13.232:80 0 60.002 200 16e2e96b5d44022d4fa91c3104781bc7
2026/07/22 09:42:10 [error] 35080#35080: *198985750 upstream timed out (110: Connection timed out) while reading upstream, client: <node-ip>, server: _, request: "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833796250&timeout=9m0s&timeoutSeconds=540&watch=true HTTP/2.0", upstream: "http://10.52.13.232:80/api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833796250&timeout=9m0s&timeoutSeconds=540&watch=true", host: "<master-vip>"
<node-ip> - - [22/Jul/2026:09:42:40 +0000] "GET /api/v1/namespaces/fleet-local/secrets?fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833796250 HTTP/2.0" 200 2650 "-" "rancher-system-agent/v0.0.0 (linux/amd64) kubernetes/$Format" 98 0.003 [cattle-system-rancher-80] [] 10.52.11.90:80 2650 0.003 200 2f2a893dbb35195e064574283fbf4cec
2026/07/22 09:43:40 [error] 35080#35080: *198985750 upstream timed out (110: Connection timed out) while reading upstream, client: <node-ip>, server: _, request: "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833798126&timeout=9m16s&timeoutSeconds=556&watch=true HTTP/2.0", upstream: "http://10.52.4.106:80/api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833798126&timeout=9m16s&timeoutSeconds=556&watch=true", host: "<master-vip>"
<node-ip> - - [22/Jul/2026:09:43:40 +0000] "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833798126&timeout=9m16s&timeoutSeconds=556&watch=true HTTP/2.0" 200 0 "-" "rancher-system-agent/v0.0.0 (linux/amd64) kubernetes/$Format" 149 60.001 [cattle-system-rancher-80] [] 10.52.4.106:80 0 60.001 200 a56a5876e376983e4fb0de4da3a5d4c6
<node-ip> - - [22/Jul/2026:09:44:14 +0000] "GET /api/v1/namespaces/fleet-local/secrets?fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833798126 HTTP/2.0" 200 2650 "-" "rancher-system-agent/v0.0.0 (linux/amd64) kubernetes/$Format" 98 0.003 [cattle-system-rancher-80] [] 10.52.13.232:80 2650 0.003 200 672999c238efb6489cfb2d2e26c92ae8
2026/07/22 09:45:14 [error] 35080#35080: *198985750 upstream timed out (110: Connection timed out) while reading upstream, client: <node-ip>, server: _, request: "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833800054&timeout=6m35s&timeoutSeconds=395&watch=true HTTP/2.0", upstream: "http://10.52.11.90:80/api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833800054&timeout=6m35s&timeoutSeconds=395&watch=true", host: "<master-vip>"
<node-ip> - - [22/Jul/2026:09:45:14 +0000] "GET /api/v1/namespaces/fleet-local/secrets?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dcustom-e711b305e5c5-machine-plan&resourceVersion=833800054&timeout=6m35s&timeoutSeconds=395&watch=true HTTP/2.0" 200 0 "-" "rancher-system-agent/v0.0.0 (linux/amd64) kubernetes/$Format" 149 60.001 [cattle-system-rancher-80] [] 10.52.11.90:80 0 60.001 200 059f6e4458e8bc1a785bfcba7b021b55
kubectl -n fleet-local get machine
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
custom-129b79bbaa9d local Provisioning 6h12m
custom-185fa781d934 local hci-**-** rke2://hci-**-** Running 253d
.
.
.
fleet-local secret custom-e711b305e5c5-machine-plan的data是空
kubectl -n fleet-local get secret | grep custom-e711b305e5c5
custom-e711b305e5c5-machine-bootstrap rke.cattle.io/bootstrap 1 6h22m
custom-e711b305e5c5-machine-bootstrap-token-ktvwb kubernetes.io/service-account-token 3 6h22m
custom-e711b305e5c5-machine-plan rke.cattle.io/machine-plan 0 6h22m
custom-e711b305e5c5-machine-plan-token-8rm2z kubernetes.io/service-account-token 3 6h22m
重建的话题吧，描述好上下文