Rancher 2.6.8 导入集群时触发已导入集群资源的处理，是为了解决什么问题？能跳过吗？

dx1 · 2024 年1 月 26 日 15:37

Rancher Server 设置

Rancher 版本：2.6.8

问题描述：
导入新集群时，会触发已存在 cluster 对象入队

2024-01-26T18:59:26.136012514+08:00 2024/01/26 10:59:26 [DEBUG] [clustermanager] creating AccessControl for cluster c-m-q5xmprp5
2024-01-26T18:59:26.395932435+08:00 goroutine 71752 [running]:
2024-01-26T18:59:26.395936423+08:00 runtime/debug.Stack()
2024-01-26T18:59:26.395939003+08:00     /opt/homebrew/opt/go/libexec/src/runtime/debug/stack.go:24 +0x5e
2024-01-26T18:59:26.395941747+08:00 runtime/debug.PrintStack()
2024-01-26T18:59:26.395944419+08:00     /opt/homebrew/opt/go/libexec/src/runtime/debug/stack.go:16 +0x13
2024-01-26T18:59:26.395947171+08:00 github.com/rancher/lasso/pkg/controller.(*sharedController).RegisterHandler.func1()
2024-01-26T18:59:26.395950135+08:00     /Users/pz/go/src/github.com/rancher/rancher/vendor/github.com/rancher/lasso/pkg/controller/sharedcontroller.go:119 +0x131
2024-01-26T18:59:26.395953781+08:00 github.com/rancher/lasso/pkg/controller.(*HandlerTransaction).Commit(0xc0083d2380?)
2024-01-26T18:59:26.395956804+08:00     /Users/pz/go/src/github.com/rancher/rancher/vendor/github.com/rancher/lasso/pkg/controller/transaction.go:37 +0xb8
2024-01-26T18:59:26.395959707+08:00 github.com/rancher/rancher/pkg/clustermanager.(*Manager).doStart.func2()
2024-01-26T18:59:26.395962581+08:00     /Users/pz/go/src/github.com/rancher/rancher/pkg/clustermanager/manager.go:244 +0x15b
2024-01-26T18:59:26.395965319+08:00 created by github.com/rancher/rancher/pkg/clustermanager.(*Manager).doStart in goroutine 71718
2024-01-26T18:59:26.395968611+08:00     /Users/pz/go/src/github.com/rancher/rancher/pkg/clustermanager/manager.go:235 +0x539

结果：

这里触发 c.EnqueueKey(key) 是出于什么考虑？可以跳过这个逻辑吗？

预期结果：

新引入集群时，不会触发已有集群入队。

截图：

pkg/controller/sharedcontroller.go:111

其他上下文信息：

日志

jacie · 2024 年1 月 29 日 02:04

请描述完整的复现步骤或提供完整的rancher日志
另外这个 c-m-q5xmprp5 的集群ID是已有的健康集群么？

dx1 · 2024 年1 月 29 日 02:17

代码变更，在 pkg/controller/sharedcontroller.go:113 添加了 debug.PrintStack()
rancher 打开 --debug 的启动参数，在集群注册时可以看到 debug creating AccessControl 日志
c-m-q5xmprp5 的集群ID 是新建导入集群时，生成的新的集群 ID

代码，stack 是在 https://github.com/rancher/lasso/blob/master/pkg/controller/sharedcontroller.go#L116 处增加了一行输出。

	getHandlerTransaction(ctx).do(func() {
		debug.PrintStack()

		s.handler.Register(ctx, name, handler)

rancher 启动参数添加了 --debug：

rancher --http-listen-port=80 --https-listen-port=443 --audit-log-path=/var/log/auditlog/rancher-api-audit.log --audit-level=0 --audit-log-maxage=10 --audit-log-maxbackup=10 --audit-log-maxsize=100 --debug --no-cacerts --h
ttp-listen-port=80 --https-listen-port=443 --add-local=true

日志

2024-01-26T18:59:26.135996207+08:00 2024/01/26 10:59:26 [DEBUG] Registering defaultSvcAccountHandler for checking default service account of system namespaces
2024-01-26T18:59:26.135999467+08:00 controller: &{0x12d3bc0 <nil> <nil> {  } 0xc010f99f90 {1 0} false <nil> 0xc008f86000}, handlers: &{0 /v1, Resource=configmaps {{0 0} 0 0 {{} 0} {{} 0}} []} 
2024-01-26T18:59:26.136001987+08:00 controller: &{0x12d3bc0 <nil> <nil> {  } 0xc028dee230 {1 0} false <nil> 0xc008f86580}, handlers: &{0 apiregistration.k8s.io/v1, Resource=apiservices {{0 0} 0 0 {{} 0} {{} 0}} []} 
2024-01-26T18:59:26.136006273+08:00 controller: &{0x12d3bc0 <nil> <nil> {  } 0xc028dee320 {1 0} false <nil> 0xc008f86840}, handlers: &{0 /v1, Resource=secrets {{0 0} 0 0 {{} 0} {{} 0}} []} 
2024-01-26T18:59:26.136008786+08:00 controller: &{0x12d3bc0 <nil> <nil> {  } 0xc028dee410 {1 0} false <nil> 0xc008f86dc0}, handlers: &{0 /v1, Resource=serviceaccounts {{0 0} 0 0 {{} 0} {{} 0}} []} 
2024-01-26T18:59:26.136012514+08:00 2024/01/26 10:59:26 [DEBUG] [clustermanager] creating AccessControl for cluster c-m-q5xmprp5
2024-01-26T18:59:26.136016128+08:00 controller: &{0x12d3bc0 <nil> <nil> {  } 0xc028dee9b0 {1 0} false <nil> 0xc008f87080}, handlers: &{0 rbac.authorization.k8s.io/v1, Resource=roles {{0 0} 0 0 {{} 0} {{} 0}} []} 
2024-01-26T18:59:26.136019813+08:00 2024/01/26 10:59:26 [INFO] Starting cluster controllers for c-m-q5xmprp5
2024-01-26T18:59:26.152213615+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, handler: snapshotbackpopulate, elapsed: 0 s
2024-01-26T18:59:26.152233366+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, all elapsed: 0 s
2024-01-26T18:59:26.152237979+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, handler: helm-app-configmap, elapsed: 0 s
2024-01-26T18:59:26.152241750+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, handler: prov-log-configmap, elapsed: 0 s
2024-01-26T18:59:26.152245608+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, handler: planner, elapsed: 0 s
2024-01-26T18:59:26.152249142+08:00 key: cattle-fleet-local-system/fleet-agent-lock, gvk: /v1, Resource=configmaps, all elapsed: 0 s
2024-01-26T18:59:26.195805718+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-deploy, elapsed: 0 s
2024-01-26T18:59:26.195868532+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-scoped-gc, elapsed: 0 s
2024-01-26T18:59:26.195874222+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-provisioner-controller, elapsed: 0 s
2024-01-26T18:59:26.195878246+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-stats, elapsed: 0 s
2024-01-26T18:59:26.195881592+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: temporary-credentials, elapsed: 0 s
2024-01-26T18:59:26.195884984+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: restrictedAdminsRBACCluster, elapsed: 0 s
2024-01-26T18:59:26.195888042+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-secret-migrator, elapsed: 0 s
2024-01-26T18:59:26.195948783+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: management-mcapp-cluster-controller, elapsed: 0 s
2024-01-26T18:59:26.195955441+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-rbac-remove, elapsed: 0 s
2024-01-26T18:59:26.195958824+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: k3s-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.195961768+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: aks-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.195964847+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: eks-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.195967380+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: gke-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.195970066+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-refresher-controller, elapsed: 0 s
2024-01-26T18:59:26.195973201+08:00 key: _all_, gvk: management.cattle.io/v3, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.195976733+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: user-controllers-controller, elapsed: 0 s
2024-01-26T18:59:26.195979827+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: global-admin-cluster-sync, elapsed: 0 s
2024-01-26T18:59:26.195982994+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: networkpolicy-deferred, elapsed: 0 s
2024-01-26T18:59:26.195985287+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: psptpb-deferred, elapsed: 0 s
2024-01-26T18:59:26.195987760+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: certs-expiration-deferred, elapsed: 0 s
2024-01-26T18:59:26.195990109+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: pspdelete-deferred, elapsed: 0 s
2024-01-26T18:59:26.196009574+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: monitoring-deferred, elapsed: 0 s
2024-01-26T18:59:26.196012521+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: prometheus-operator-deferred, elapsed: 0 s
2024-01-26T18:59:26.196051095+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: kubernetes-provider, elapsed: 0 s
2024-01-26T18:59:26.196055470+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-provisioning-operator, elapsed: 0 s
2024-01-26T18:59:26.196058622+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-registration-token-trigger, elapsed: 0 s
2024-01-26T18:59:26.196061392+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-v1-trigger, elapsed: 0 s
2024-01-26T18:59:26.196067722+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: provisioning-cluster-create, elapsed: 0 s
2024-01-26T18:59:26.196405240+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: provisioning-cluster-create, elapsed: 0 s
2024-01-26T18:59:26.196415168+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-watch, elapsed: 0 s
2024-01-26T18:59:26.196474089+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-watch, elapsed: 0 s
2024-01-26T18:59:26.196483540+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.196486493+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: fleet-cluster-assign, elapsed: 0 s
2024-01-26T18:59:26.196489244+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: rke-control-plane-trigger, elapsed: 0 s
2024-01-26T18:59:26.196674660+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-rbac-delete, elapsed: 0 s
2024-01-26T18:59:26.196710551+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-agent-controller-cleanup, elapsed: 0 s
2024-01-26T18:59:26.196715549+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: certificate-expiration, elapsed: 0 s
2024-01-26T18:59:26.196718482+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: clusterCreateUpdate, elapsed: 0 s
2024-01-26T18:59:26.196843944+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-deploy, elapsed: 0 s
2024-01-26T18:59:26.196851947+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-scoped-gc, elapsed: 0 s
2024-01-26T18:59:26.197005257+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-provisioner-controller, elapsed: 0 s
2024-01-26T18:59:26.197012389+08:00 key: c-m-q5xmprp5, controller name: management.cattle.io/v3, Resource=clusters, handler name: cluster-provisioner-controller, err: ignore handler error
2024-01-26T18:59:26.204211079+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cluster-create, elapsed: 0 s
2024-01-26T18:59:26.204237278+08:00 2024/01/26 10:59:26 [DEBUG] DesiredSet - No change(2) management.cattle.io/v3, Kind=Cluster /c-m-q5xmprp5 for cluster-create dxzq-238/dxzq-238
2024-01-26T18:59:26.204241885+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cluster-create, elapsed: 0 s
2024-01-26T18:59:26.204245217+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: provisioning-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.204262859+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: fleet-cluster, elapsed: 0 s
2024-01-26T18:59:26.204266010+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: fleet-cluster, elapsed: 0 s
2024-01-26T18:59:26.204269354+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster, elapsed: 0 s
2024-01-26T18:59:26.204272260+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster, elapsed: 0 s
2024-01-26T18:59:26.204275037+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.204277932+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-agent, elapsed: 0 s
2024-01-26T18:59:26.204280944+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-agent, elapsed: 0 s
2024-01-26T18:59:26.204286607+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.204289430+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.204306032+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: harvester-secret-migrator, elapsed: 0 s
2024-01-26T18:59:26.204309035+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cloud-config-secret-remover, elapsed: 0 s
2024-01-26T18:59:26.204311845+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: auth-prov-v2-cluster, elapsed: 0 s
2024-01-26T18:59:26.204315208+08:00 key: dxzq-238/dxzq-238, gvk: provisioning.cattle.io/v1, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.204318450+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: provisioning-cluster-trigger, elapsed: 0 s
2024-01-26T18:59:26.204322680+08:00 key: c-m-q5xmprp5/default-token, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, handler: cluster-registration-token, elapsed: 0 s
2024-01-26T18:59:26.204326410+08:00 key: c-m-q5xmprp5/default-token, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, all elapsed: 0 s
2024-01-26T18:59:26.204329220+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: planner, elapsed: 0 s
2024-01-26T18:59:26.204332480+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: rke-control-plane, elapsed: 0 s
2024-01-26T18:59:26.204335417+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: rke-control-plane-remove, elapsed: 0 s
2024-01-26T18:59:26.204347398+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: monitor-system-upgrade-controller-readiness, elapsed: 0 s
2024-01-26T18:59:26.204350501+08:00 key: dxzq-238/dxzq-238, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, all elapsed: 0 s
2024-01-26T18:59:26.219133875+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-stats, elapsed: 0 s
2024-01-26T18:59:26.219157521+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: temporary-credentials, elapsed: 0 s
2024-01-26T18:59:26.219162296+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: restrictedAdminsRBACCluster, elapsed: 0 s
2024-01-26T18:59:26.219166045+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.219184106+08:00 2024/01/26 10:59:26 [DEBUG] waiting for cluster [c-m-dqv2q4ph] agent to connect
2024-01-26T18:59:26.226070495+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-secret-migrator, elapsed: 0 s
2024-01-26T18:59:26.226100748+08:00 key: c-m-q5xmprp5, controller name: management.cattle.io/v3, Resource=clusters, handler name: cluster-secret-migrator, err: Operation cannot be fulfilled on clusters.management.cattle.io "c-m-q5xmprp5": the object has been modified; please apply your changes to the latest version and try again
2024-01-26T18:59:26.226106610+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: management-mcapp-cluster-controller, elapsed: 0 s
2024-01-26T18:59:26.226117783+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-rbac-remove, elapsed: 0 s
2024-01-26T18:59:26.226121001+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: k3s-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.226126335+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: aks-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.226129300+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: eks-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.226132334+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: gke-operator-controller, elapsed: 0 s
2024-01-26T18:59:26.226135229+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-refresher-controller, elapsed: 0 s
2024-01-26T18:59:26.226138660+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.226142024+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: user-controllers-controller, elapsed: 0 s
2024-01-26T18:59:26.226144951+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: global-admin-cluster-sync, elapsed: 0 s
2024-01-26T18:59:26.226147730+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: networkpolicy-deferred, elapsed: 0 s
2024-01-26T18:59:26.226150716+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: psptpb-deferred, elapsed: 0 s
2024-01-26T18:59:26.226153804+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: certs-expiration-deferred, elapsed: 0 s
2024-01-26T18:59:26.226156468+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: pspdelete-deferred, elapsed: 0 s
2024-01-26T18:59:26.226159419+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: monitoring-deferred, elapsed: 0 s
2024-01-26T18:59:26.226793243+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: prometheus-operator-deferred, elapsed: 0 s
2024-01-26T18:59:26.226810906+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: kubernetes-provider, elapsed: 0 s
2024-01-26T18:59:26.226814404+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-provisioning-operator, elapsed: 0 s
2024-01-26T18:59:26.226817592+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-registration-token-trigger, elapsed: 0 s
2024-01-26T18:59:26.226826104+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-v1-trigger, elapsed: 0 s
2024-01-26T18:59:26.226829243+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: provisioning-cluster-create, elapsed: 0 s
2024-01-26T18:59:26.226831866+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: provisioning-cluster-create, elapsed: 0 s
2024-01-26T18:59:26.226839058+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-watch, elapsed: 0 s
2024-01-26T18:59:26.226841874+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-watch, elapsed: 0 s
2024-01-26T18:59:26.226844803+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.226847574+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: fleet-cluster-assign, elapsed: 0 s
2024-01-26T18:59:26.226850787+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: rke-control-plane-trigger, elapsed: 0 s
2024-01-26T18:59:26.226854226+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: mgmt-cluster-rbac-delete, elapsed: 0 s
2024-01-26T18:59:26.226857526+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: cluster-agent-controller-cleanup, elapsed: 0 s
2024-01-26T18:59:26.226860241+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: certificate-expiration, elapsed: 0 s
2024-01-26T18:59:26.226864216+08:00 key: c-m-f6z9fqxl, gvk: management.cattle.io/v3, Resource=clusters, handler: clusterCreateUpdate, elapsed: 0 s
2024-01-26T18:59:26.226867142+08:00 2024/01/26 10:59:26 [INFO] Stopping cluster agent for c-m-dqv2q4ph
2024-01-26T18:59:26.226870544+08:00 2024/01/26 10:59:26 [ERROR] failed to start cluster controllers c-m-dqv2q4ph: context canceled
2024-01-26T18:59:26.226874096+08:00 2024/01/26 10:59:26 [DEBUG] No active connection for cluster [c-m-f6z9fqxl], will wait for about 30 seconds
2024-01-26T18:59:26.231786262+08:00 key: c-m-f6z9fqxl/system, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, handler: cluster-registration-token, elapsed: 0 s
2024-01-26T18:59:26.231801158+08:00 key: c-m-f6z9fqxl/system, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, all elapsed: 0 s
2024-01-26T18:59:26.231805505+08:00 key: c-m-f6z9fqxl/default-token, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, handler: cluster-registration-token, elapsed: 0 s
2024-01-26T18:59:26.231809751+08:00 key: c-m-f6z9fqxl/default-token, gvk: management.cattle.io/v3, Resource=clusterregistrationtokens, all elapsed: 0 s
2024-01-26T18:59:26.231821925+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: provisioning-cluster-trigger, elapsed: 0 s
2024-01-26T18:59:26.231843050+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: planner, elapsed: 0 s
2024-01-26T18:59:26.231849177+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: rke-control-plane, elapsed: 0 s
2024-01-26T18:59:26.231852496+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: rke-control-plane-remove, elapsed: 0 s
2024-01-26T18:59:26.231856035+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, handler: monitor-system-upgrade-controller-readiness, elapsed: 0 s
2024-01-26T18:59:26.231859450+08:00 key: dxzq-195/dxzq-195, gvk: rke.cattle.io/v1, Resource=rkecontrolplanes, all elapsed: 0 s
2024-01-26T18:59:26.231863072+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cluster-create, elapsed: 0 s
2024-01-26T18:59:26.233087986+08:00 2024/01/26 10:59:26 [DEBUG] DesiredSet - No change(2) management.cattle.io/v3, Kind=Cluster /c-m-f6z9fqxl for cluster-create dxzq-195/dxzq-195
2024-01-26T18:59:26.234163616+08:00 2024/01/26 10:59:26 [DEBUG] DesiredSet - No change(2) /v1, Kind=Secret dxzq-195/dxzq-195-kubeconfig for cluster-create dxzq-195/dxzq-195
2024-01-26T18:59:26.235204543+08:00 2024/01/26 10:59:26 [DEBUG] DesiredSet - No change(2) management.cattle.io/v3, Kind=ClusterRoleTemplateBinding c-m-f6z9fqxl/c-m-f6z9fqxl-dxzq-195-owner for cluster-create dxzq-195/dxzq-195
2024-01-26T18:59:26.235686432+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cluster-create, elapsed: 0 s
2024-01-26T18:59:26.235697006+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: provisioning-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.235703300+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: fleet-cluster, elapsed: 0 s
2024-01-26T18:59:26.235727196+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: fleet-cluster, elapsed: 0 s
2024-01-26T18:59:26.235731601+08:00 key: dxzq-195/dxzq-195, controller name: provisioning.cattle.io/v1, Resource=clusters, handler name: fleet-cluster, err: ignore handler error
2024-01-26T18:59:26.235734682+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster, elapsed: 0 s
2024-01-26T18:59:26.235857687+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster, elapsed: 0 s
2024-01-26T18:59:26.235864040+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: rke-cluster-remove, elapsed: 0 s
2024-01-26T18:59:26.235867571+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-agent, elapsed: 0 s
2024-01-26T18:59:26.235966864+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-agent, elapsed: 0 s
2024-01-26T18:59:26.235973522+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.236048966+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: manage-system-upgrade-controller, elapsed: 0 s
2024-01-26T18:59:26.236054268+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: harvester-secret-migrator, elapsed: 0 s
2024-01-26T18:59:26.236057538+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: cloud-config-secret-remover, elapsed: 0 s
2024-01-26T18:59:26.236065279+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, handler: auth-prov-v2-cluster, elapsed: 0 s
2024-01-26T18:59:26.236068611+08:00 key: dxzq-195/dxzq-195, gvk: provisioning.cattle.io/v1, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.239227269+08:00 start controller: [management.cattle.io/v3, Kind=SamlToken], workers: 50
2024-01-26T18:59:26.239241397+08:00 2024/01/26 10:59:26 [INFO] Starting management.cattle.io/v3, Kind=SamlToken controller
2024-01-26T18:59:26.239245495+08:00 run controller: [management.cattle.io/v3, Kind=SamlToken], workers: 50, workqueue: &{0xc014d5db00 0xc0197af248}
2024-01-26T18:59:26.249245944+08:00 key: c-m-q5xmprp5, gvk: management.cattle.io/v3, Resource=clusters, all elapsed: 0 s
2024-01-26T18:59:26.332669975+08:00 2024/01/26 10:59:26 [DEBUG] waiting for cluster [c-m-4zmrg92d] agent to connect