Docker离线安装单节点rancher capi-controller-manager报错

hxp0176 · 2025 年12 月 22 日 06:29

Rancher Server 设置

Rancher 版本：2.13.0
安装选项: Docker install
安装命令：docker run -d --restart=unless-stopped -p 12980:80 -p 12443:443 --privileged -v /root/rancher-registries.yaml:/etc/rancher/k3s/registries.yaml:ro -e CATTLE_SYSTEM_DEFAULT_REGISTRY=192.168.22.214:5000 -e CATTLE_SYSTEM_CATALOG=bundled --name rancher 192.168.22.214:5000/rancher/rancher:v2.13.0
- 如果是 Helm Chart 安装，需要提供 Local 集群的类型（RKE1, RKE2, k3s, EKS, 等）和版本：无
在线或离线部署：离线部署

下游集群信息

Kubernetes 版本: v1.34.1+k3s1
Cluster Type (Local/Downstream): Local
- 如果 Downstream，是什么类型的集群?(自定义/导入或为托管等):

用户信息

登录用户的角色是什么？（管理员/集群所有者/集群成员/项目所有者/项目成员/自定义）：管理员
- 如果自定义，自定义权限集：

主机操作系统： openEuler 20.03SP4 x64

问题描述：
通过docker离线安装，离线镜像来源： Rancher Releases Mirror，安装时必定经历一次内置k3s的FATAL报错，容器自动重启，重启后正常运行，登录rancher后台后，查看到capi-controller-manager这个pod拉取不到镜像报ImagePullBackOff ，这个镜像无法通过官网文档所提供的离线自建docker registry的方式安装，不过看起来也不影响使用。

重现步骤：
docker环境离线安装2.13.0版本rancher
结果：
首次启动必定出现一次FATAL并自动重启，重启后服务恢复正常
预期结果：
正常启动并不产生报错
截图：

其他上下文信息：

日志

hxp0176 · 2025 年12 月 22 日 07:55

补一部分首次启动必崩溃的日志

2025/12/22 07:53:31 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=kube-node-lease, err=Operation cannot be fulfilled on namespaces "kube-node-lease": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:31 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=kube-node-lease
I1222 07:53:31.310007      43 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
2025/12/22 07:53:31 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found
2025/12/22 07:53:31 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=DigitaloceanConfig controller
2025/12/22 07:53:31 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate
W1222 07:53:31.390291      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:31 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-local-user-passwords
2025/12/22 07:53:31 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace local
2025/12/22 07:53:31 [INFO] uploading digitaloceancredentialConfig to credentialconfig schema
2025/12/22 07:53:31 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=kube-system
2025/12/22 07:53:31 [INFO] Starting rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate controller
2025/12/22 07:53:31 [ERROR] error syncing 'cluster/fleet-local/local': handler auth-prov-v2-roletemplate: no matches for kind "VmwarevsphereConfig" in version "rke-machine-config.cattle.io/v1", requeuing
W1222 07:53:31.599412      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:31 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachine
2025/12/22 07:53:31 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-system": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:31 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=fleet-default
2025/12/22 07:53:31 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace local
2025/12/22 07:53:31 [INFO] imperative api APIService cert updated
2025/12/22 07:53:31 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
2025/12/22 07:53:31 [INFO] uploading vmwarevspherecredentialConfig to credentialconfig schema
2025/12/22 07:53:31 [INFO] Starting rke-machine.cattle.io/v1, Kind=DigitaloceanMachine controller
2025/12/22 07:53:31 [ERROR] error syncing 'cluster/fleet-local/local': handler auth-prov-v2-roletemplate: no matches for kind "VmwarevsphereConfig" in version "rke-machine-config.cattle.io/v1", requeuing
W1222 07:53:31.817786      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:31 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate
2025/12/22 07:53:31 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=fleet-default, err=Operation cannot be fulfilled on namespaces "fleet-default": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:31 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-qkp4j
2025/12/22 07:53:31 [INFO] uploading amazonec2credentialConfig to credentialconfig schema
2025/12/22 07:53:31 [INFO] Starting rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate controller
2025/12/22 07:53:31 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig
W1222 07:53:31.935247      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:32 [INFO] Active TLS secret cattle-system/serving-cert (ver=1531) (count 5): map[field.cattle.io/projectId:local:p-xsqd8 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.17.0.2:172.17.0.2 listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=2CB79000E12CA538E287BB34AC8563151F6AC1EC]
2025/12/22 07:53:32 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace p-qkp4j
2025/12/22 07:53:32 [INFO] uploading digitaloceancredentialConfig to credentialconfig schema
2025/12/22 07:53:32 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig controller
2025/12/22 07:53:32 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine
W1222 07:53:32.342771      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:32 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=kube-system
2025/12/22 07:53:32 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=fleet-local
I1222 07:53:32.495332      43 warnings.go:110] "Warning: v1 ComponentStatus is deprecated in v1.19+"
2025/12/22 07:53:32 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-xsqd8
2025/12/22 07:53:32 [INFO] Starting rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine controller
2025/12/22 07:53:32 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=Amazonec2Config
W1222 07:53:32.551074      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:32 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace p-xsqd8
2025/12/22 07:53:32 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "fleet-local": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:32 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig
2025/12/22 07:53:32 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachine
2025/12/22 07:53:32 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-system
2025/12/22 07:53:32 [INFO] Active TLS secret cattle-system/tls-rancher-internal (ver=1552) (count 2): map[field.cattle.io/projectId:local:p-xsqd8 listener.cattle.io/cn-10.43.143.193:10.43.143.193 listener.cattle.io/fingerprint:SHA1=DF9B9CF16BDC0851659C3EB41E9355934FDAA56F]
2025/12/22 07:53:32 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=Amazonec2Config controller
2025/12/22 07:53:32 [ERROR] Failed to read API for groups map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:32 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate
2025/12/22 07:53:33 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-system": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:33 [INFO] Starting rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate controller
2025/12/22 07:53:33 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2Machine
W1222 07:53:33.181213      43 gvks.go:80] Failed to read API for groups: map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:33 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=fleet-local
2025/12/22 07:53:33 [INFO] Starting rke-machine.cattle.io/v1, Kind=Amazonec2Machine controller
2025/12/22 07:53:33 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate
2025/12/22 07:53:34 [INFO] Starting rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate controller
2025/12/22 07:53:34 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
2025/12/22 07:53:34 [INFO] Skipping handler for clusterrepo rancher-rke2-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-12-22 08:53:33 +0000 UTC, now is 2025-12-22 07:53:34.790118934 +0000 UTC
2025/12/22 07:53:34 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
I1222 07:53:35.009772      43 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
2025/12/22 07:53:35 [INFO] Skipping handler for clusterrepo rancher-rke2-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-12-22 08:53:33 +0000 UTC, now is 2025-12-22 07:53:35.00958152 +0000 UTC
2025/12/22 07:53:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-system
I1222 07:53:35.275186      43 warnings.go:110] "Warning: v1 ComponentStatus is deprecated in v1.19+"
2025/12/22 07:53:35 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:53:35 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachine
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2Machine
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=Amazonec2Config
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachine
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachineTemplate
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=DigitaloceanConfig
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=AzureConfig
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=LinodeConfig
2025/12/22 07:53:35 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachine
2025/12/22 07:53:35 [ERROR] Failed to read API for groups map[ext.cattle.io/v1:stale GroupVersion discovery: ext.cattle.io/v1]
2025/12/22 07:53:35 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
2025/12/22 07:53:36 [INFO] EnsureSecretForServiceAccount: waiting for secret [cattle-impersonation-system:cattle-impersonation-u-mo773yttt4-token-d684s] for service account [cattle-impersonation-system:cattle-impersonation-u-mo773yttt4] to be populated with token
2025/12/22 07:53:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-impersonation-system
2025/12/22 07:53:36 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found
2025/12/22 07:53:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-impersonation-system": the object has been modified; please apply your changes to the latest version and try again
I1222 07:53:36.368619      43 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
2025/12/22 07:53:36 [INFO] EnsureSecretForServiceAccount: got the service account token for service account [cattle-impersonation-system:cattle-impersonation-u-mo773yttt4] in 50.561741ms
2025/12/22 07:53:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-impersonation-system
I1222 07:53:36.412757      43 warnings.go:110] "Warning: v1 ComponentStatus is deprecated in v1.19+"
2025/12/22 07:53:36 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate
2025/12/22 07:53:37 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
2025/12/22 07:53:37 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole- for corresponding GlobalRole
2025/12/22 07:53:40 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:53:40 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:53:41 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found
2025/12/22 07:53:43 [INFO] initializing required info for telemetry manager...
2025/12/22 07:53:45 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:53:45 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:53:45 [INFO] telemetry manager info not available yet, re-queing check...
2025/12/22 07:53:46 [INFO] Skipping handler for clusterrepo rancher-charts because the resource version has changed from 897 to 1676
2025/12/22 07:53:46 [INFO] Skipping handler for clusterrepo rancher-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-12-22 08:53:45 +0000 UTC, now is 2025-12-22 07:53:46.311124375 +0000 UTC
2025/12/22 07:53:46 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-fleet-system
2025/12/22 07:53:46 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-fleet-system": the object has been modified; please apply your changes to the latest version and try again
2025/12/22 07:53:46 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-xsqd8 to namespace=cattle-fleet-system
2025/12/22 07:53:48 [INFO] Skipping handler for clusterrepo rancher-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-12-22 08:53:45 +0000 UTC, now is 2025-12-22 07:53:48.108166744 +0000 UTC
2025/12/22 07:53:50 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:53:50 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:53:55 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:53:55 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:53:58 [INFO] Skipping handler for clusterrepo rancher-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-12-22 08:53:45 +0000 UTC, now is 2025-12-22 07:53:58.00740491 +0000 UTC
2025/12/22 07:54:00 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:54:00 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:54:00 [INFO] initializing required info for telemetry manager...
2025/12/22 07:54:00 [INFO] telemetry manager info not available yet, re-queing check...
2025/12/22 07:54:05 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:54:05 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:54:10 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:54:10 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:54:15 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:54:15 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:54:15 [INFO] initializing required info for telemetry manager...
2025/12/22 07:54:20 [INFO] RDPClient: Checking if dialer is built...
2025/12/22 07:54:20 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/22 07:54:20 [ERROR] watcher channel closed
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): pod cattle-system/helm-operation-tqz5h failed, watch closed
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
E1222 07:54:21.005210      43 leaderelection.go:441] Failed to update lock optimistically: Put "https://127.0.0.1:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cattle-controllers?timeout=15m0s": read tcp 127.0.0.1:39798->127.0.0.1:6443: read: connection reset by peer, falling back to slow path
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
E1222 07:54:21.005593      43 leaderelection.go:448] error retrieving resource lock kube-system/cattle-controllers: Get "https://127.0.0.1:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cattle-controllers?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
E1222 07:54:21.005932      43 leaderelection.go:441] Failed to update lock optimistically: Put "https://127.0.0.1:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cattle-controllers?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused, falling back to slow path
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
E1222 07:54:21.006221      43 leaderelection.go:448] error retrieving resource lock kube-system/cattle-controllers: Get "https://127.0.0.1:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cattle-controllers?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet (release name: fleet): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [ERROR] Failed to install system chart fleet-crd (release name: fleet-crd): Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
2025/12/22 07:54:21 [FATAL] k3s exited with: exit status 1

hxp0176 · 2025 年12 月 23 日 01:27

即使将registry.k8s.io指向本地registry，也会由于注册表不同，rancher拉取的是registry.k8s.io/cluster-api/cluster-api-controller，无法正常拉取到所提供的包中已有的cluster-api-controller镜像，同时观察到在缺失这个pod的情况下，rancher会在数个小时后崩溃且不可恢复，docker logs看到输出如下

INFO: Running k3s server --cluster-init --cluster-reset
2025/12/23 00:09:32 [INFO] Rancher version v2.13.0 (f94ac947f75e312f1ab9217d21b2770b48b734c8) is starting
2025/12/23 00:09:32 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:false AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLogLevel:0 AuditLogEnabled:false Features: ClusterRegistry: AggregationRegistrationTimeout:5m0s}
2025/12/23 00:09:32 [INFO] Listening on /tmp/log.sock
2025/12/23 00:09:32 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:34 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:36 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:38 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:40 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:42 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:44 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:46 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:48 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6444/version?timeout=15m0s": dial tcp 127.0.0.1:6444: connect: connection refused
2025/12/23 00:09:51 [INFO] Running in single server mode, will not peer connections
2025/12/23 00:09:51 [INFO] Scanning NodeTemplates in namespace: cattle-global-nt, group: nodetemplates.management.cattle.io
2025/12/23 00:09:51 [INFO] Scanning ClusterTemplates in namespace: cattle-global-data, group: clustertemplates.management.cattle.io
2025/12/23 00:09:51 [INFO] [deferred-capi - WaitForClient] waiting for CAPI CRDs to be established...
2025/12/23 00:09:51 [INFO] [deferred-ext] WaitForClient starting waiter for EXT api-service availability
I1223 00:09:51.875306     152 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
2025/12/23 00:09:51 [INFO] Applying CRD features.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD auditpolicies.auditlog.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD clusterrepos.catalog.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD clusters.provisioning.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD custommachines.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD etcdsnapshots.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD rkebootstraps.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD rkebootstraptemplates.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD rkeclusters.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD rkecontrolplanes.rke.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD clusterroletemplatebindings.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD dynamicschemas.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD globalroles.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD globalrolebindings.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD nodedrivers.management.cattle.io
2025/12/23 00:09:58 [INFO] Updating embedded CRD projects.management.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD projectroletemplatebindings.management.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD roletemplates.management.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD users.management.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD clusterproxyconfigs.management.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD uiplugins.catalog.cattle.io
2025/12/23 00:09:59 [INFO] Updating embedded CRD secretrequests.telemetry.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD plans.upgrade.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD navlinks.ui.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD podsecurityadmissionconfigurationtemplates.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD clusters.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD apiservices.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD clusterregistrationtokens.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD settings.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD preferences.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD features.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD operations.catalog.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD apps.catalog.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD fleetworkspaces.management.cattle.io
2025/12/23 00:09:59 [INFO] Applying CRD managedcharts.management.cattle.io
2025/12/23 00:09:59 [INFO] Configuring auth server API body limit to 1048576 bytes
2025/12/23 00:09:59 [INFO] creating imperative extension apiserver resources
2025/12/23 00:09:59 [INFO] RDPClient: Checking if dialer is built...
2025/12/23 00:09:59 [INFO] RDPClient: Dialer is not built yet, waiting 5 secs to re-check.
2025/12/23 00:09:59 [INFO] starting imperative api cert rotator
2025/12/23 00:09:59 [INFO] imperative api APIService cert updated
2025/12/23 00:10:00 [INFO] Successfully installed useractivity store
2025/12/23 00:10:00 [INFO] Successfully installed token store
2025/12/23 00:10:00 [INFO] Successfully installed kubeconfig store
2025/12/23 00:10:00 [INFO] Successfully installed passwordchangerequest store
2025/12/23 00:10:00 [INFO] Successfully installed groupmembershiprefreshrequest store
2025/12/23 00:10:00 [INFO] Successfully installed selfuser store
I1223 00:10:00.663312     152 handler.go:285] Adding GroupVersion ext.cattle.io v1 to ResourceManager
I1223 00:10:02.276068     152 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController
I1223 00:10:02.276131     152 shared_informer.go:349] "Waiting for caches to sync" controller="RequestHeaderAuthRequestController"
I1223 00:10:02.276200     152 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I1223 00:10:02.276233     152 shared_informer.go:349] "Waiting for caches to sync" controller="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I1223 00:10:02.276539     152 secure_serving.go:211] Serving securely on 0.0.0.0:6666
I1223 00:10:02.276876     152 tlsconfig.go:243] "Starting DynamicServingCertificateController"
2025/12/23 00:10:02 [FATAL] Internal error occurred: failed calling webhook "rancher.cattle.io.namespaces.create-non-kubesystem": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/validation/namespaces?timeout=10s": no endpoints available for service "rancher-webhook"

hxp0176 · 2025 年12 月 23 日 01:34

好吧刚刚看到rancher2.13.1给离线安装的脚本增加了缺失的两个capi镜像，使用2.13.1版本应该就可以正常离线安装了，我稍后测试下

ksd · 2025 年12 月 23 日 09:57

可参考：Rancher Turtles 国内安装如何拉取镜像呢？ - #4，来自 ksd