Rancher 版本:2.7.6
安装选项 (Docker install/Helm Chart): Docker install
如果是 Helm Chart 安装,需要提供 Local 集群的类型(RKE1, RKE2, k3s, EKS, 等)和版本:k3s
在线或离线部署:
在线docker-compose部署
下游集群信息
Kubernetes 版本: 1.30
Cluster Type (Local/Downstream):
如果 Downstream,是什么类型的集群?(自定义/导入或为托管 等): 自定义
用户信息 admin
登录用户的角色是什么? (管理员/集群所有者/集群成员/项目所有者/项目成员/自定义):管理员
如果自定义,自定义权限集:Default Admin
主机操作系统: Amazon-linux 2023
问题描述:
local集群正常,下游集群一直无法注册进rancher-server,界面显示[Disconnected] Cluster agent is not connected
server端日志
rancher | 2025/02/26 03:22:19 [ERROR] error syncing 'c-m-2gdnwxzd/machine-6pblw': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-6pblw": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-6pblw, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: cc3706d9-3940-46be-9f36-ece5dfba186d, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-c75qs': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-c75qs": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-c75qs, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f184c0c2-4cc0-4b05-8e7f-2a2bf07b6fcb, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-ckm8q': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-ckm8q": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-ckm8q, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e172a685-b2d4-4839-92ed-223750cd208d, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-cphdf': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-cphdf": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-cphdf, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 153f5d09-9dba-4d10-889e-1893dbd951e6, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-gj2n5': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-gj2n5": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-gj2n5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a5fd3747-6e3e-4c53-8a20-8fa156feb61c, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-ksfz9': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-ksfz9": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-ksfz9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: aad54221-a917-4c88-809b-6f85d122264b, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-qhxw5': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-qhxw5": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-qhxw5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 0ebf5821-6584-4a00-a9ee-5701cf472810, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:20 [ERROR] error syncing 'c-m-2gdnwxzd/machine-xrgqx': handler node-controller-sync: Operation cannot be fulfilled on nodes.management.cattle.io "machine-xrgqx": StorageError: invalid object, Code: 4, Key: /registry/management.cattle.io/nodes/c-m-2gdnwxzd/machine-xrgqx, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b1a387a8-a8d4-4805-bd3e-cb87e04be32d, UID in object meta: , requeuing
rancher | 2025/02/26 03:22:22 [ERROR] error syncing 'p-qggff/creator-project-owner': handler mgmt-auth-prtb-controller: projects.management.cattle.io "c-m-2gdnwxzd/p-qggff" not found, requeuing
rancher | 2025/02/26 03:22:22 [ERROR] error syncing 'p-hrtg9/creator-project-owner': handler mgmt-auth-prtb-controller: projects.management.cattle.io "c-m-2gdnwxzd/p-hrtg9" not found, requeuing
rancher | 2025/02/26 03:22:25 [INFO] [mgmt-auth-prtb-controller] Updating owner label for roleBinding crb-6t3x5zmwjp
rancher | 2025/02/26 03:22:25 [INFO] [mgmt-auth-prtb-controller] Deleting roleBinding crb-6t3x5zmwjp
agent端日志
INFO: Environment: CATTLE_ADDRESS=192.168.7.174 CATTLE_CA_CHECKSUM= CATTLE_CLUSTER=true CATTLE_CLUSTER_AGENT_PORT=tcp://172.20.44.34:80 CATTLE_CLUSTER_AGENT_PORT_443_TCP=tcp://172.20.44.34:443 CATTLE_CLUSTER_AGENT_PORT_443_TCP_ADDR=172.20.44.34 CATTLE_CLUSTER_AGENT_PORT_443_TCP_PORT=443 CATTLE_CLUSTER_AGENT_PORT_443_TCP_PROTO=tcp CATTLE_CLUSTER_AGENT_PORT_80_TCP=tcp://172.20.44.34:80 CATTLE_CLUSTER_AGENT_PORT_80_TCP_ADDR=172.20.44.34 CATTLE_CLUSTER_AGENT_PORT_80_TCP_PORT=80 CATTLE_CLUSTER_AGENT_PORT_80_TCP_PROTO=tcp CATTLE_CLUSTER_AGENT_SERVICE_HOST=172.20.44.34 CATTLE_CLUSTER_AGENT_SERVICE_PORT=80 CATTLE_CLUSTER_AGENT_SERVICE_PORT_HTTP=80 CATTLE_CLUSTER_AGENT_SERVICE_PORT_HTTPS_INTERNAL=443 CATTLE_CLUSTER_REGISTRY= CATTLE_INGRESS_IP_DOMAIN=sslip.io CATTLE_INSTALL_UUID=e38f6482-f19a-4384-82af-6fb2e0c301ac CATTLE_INTERNAL_ADDRESS= CATTLE_IS_RKE=false CATTLE_K8S_MANAGED=true CATTLE_NODE_NAME=cattle-cluster-agent-6c76b4df48-9q8xb CATTLE_RANCHER_WEBHOOK_MIN_VERSION= CATTLE_RANCHER_WEBHOOK_VERSION=2.0.5+up0.3.5 CATTLE_SERVER=https://rancher.plt.corp.certik.com CATTLE_SERVER_VERSION=v2.7.6
INFO: Using resolv.conf: search cattle-system.svc.cluster.local svc.cluster.local cluster.local ec2.internal nameserver 172.20.0.10 options ndots:5
INFO: https://rancher.plt.corp.certik.com/ping is accessible
INFO: rancher.plt.corp.certik.com resolves to 18.214.199.108 34.237.147.16
time="2025-02-26T03:20:43Z" level=info msg="Listening on /tmp/log.sock"
time="2025-02-26T03:20:43Z" level=info msg="Rancher agent version v2.7.6 is starting"
time="2025-02-26T03:20:43Z" level=info msg="EnsureSecretForServiceAccount: waiting for secret [cattle-token-fbrnl] to be populated with token"
time="2025-02-26T03:20:43Z" level=info msg="Connecting to wss://rancher.plt.corp.certik.com/v3/connect/register with token starting with klfm96tzlcr2b9z8rjw6jvtnjdf"
time="2025-02-26T03:20:43Z" level=info msg="Connecting to proxy" url="wss://rancher.plt.corp.certik.com/v3/connect/register"