k3s版本：v1.18.8
docker版本：18.09.0
rancher版本：2.4.8
服务器配置：centos7.6 硬件2c 8g 200g
问题现象：
rancher是使用K3S方式部署的，rancher以引入外部集群的方式管理了一个小型K8S集群（3个节点），在使用的过程中经常发生rancher自动重启的情况，通过分析日志发现，操作系统不定期出发oom，系统oomkiller导致K3S重启,从而导致rancher不断重启的情况,但通过跟踪系统内存使用情况,系统内存使用率一直处于60%左右,针对该情况,请帮忙提供一些解决思路参考,谢谢
K3S重启导致k3s中服务频繁重启截图
oom日志
你发的“OOM日志”部分，如何关联到操作系统不定期出发oom，系统oomkiller导致K3S重启？
可以提供更多信息？
补充OOM日志
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.685836 63607 trace.go:116] Trace[1798892508]: "cacher list" type:*unstructured.Unstructured (started: 2022-07-28 07:32:38.043740949 +0800 CST m=+11.538462772) (total time: 642.076714ms):
Jul 28 07:32:38 rancher01 k3s: Trace[1798892508]: [642.045771ms] [642.045771ms] Ready
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.686864 63607 trace.go:116] Trace[1668289325]: "List" url:/apis/management.cattle.io/v3/clusterroletemplatebindings,user-agent:agent/v0.0.0 (linux/amd64) kubernetes/$Format,client:10.42.0.94 (started: 2022-07-28 07:32:38.043685795 +0800 CST m=+11.538407849) (total time: 643.141654ms):
Jul 28 07:32:38 rancher01 k3s: Trace[1668289325]: [642.195087ms] [642.151747ms] Listing from storage done
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.754767 63607 trace.go:116] Trace[1994546640]: "List etcd3" key:/management.cattle.io/projectnetworkpolicies,resourceVersion:,limit:10000,continue: (started: 2022-07-28 07:32:38.243271936 +0800 CST m=+11.737993783) (total time: 511.466885ms):
Jul 28 07:32:38 rancher01 k3s: Trace[1994546640]: [511.466885ms] [511.466885ms] END
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.768471 63607 trace.go:116] Trace[362802355]: "cacher list" type:*unstructured.Unstructured (started: 2022-07-28 07:32:38.241631402 +0800 CST m=+11.736353243) (total time: 526.815348ms):
Jul 28 07:32:38 rancher01 k3s: Trace[362802355]: [526.801522ms] [526.801522ms] Ready
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.768604 63607 trace.go:116] Trace[381150600]: "List" url:/apis/management.cattle.io/v3/projectnetworkpolicies,user-agent:rancher/v0.0.0 (linux/amd64) kubernetes/$Format,client:10.42.0.88 (started: 2022-07-28 07:32:38.241569387 +0800 CST m=+11.736291225) (total time: 527.010248ms):
Jul 28 07:32:38 rancher01 k3s: Trace[381150600]: [526.920887ms] [526.86781ms] Listing from storage done
Jul 28 07:32:38 rancher01 k3s: time="2022-07-28T07:32:38.777351860+08:00" level=info msg="Connecting to proxy" url="wss://172.18.30.79:6443/v1-k3s/connect"
Jul 28 07:32:38 rancher01 k3s: time="2022-07-28T07:32:38.786530820+08:00" level=info msg="Handling backend connection request [rancher01]"
Jul 28 07:32:38 rancher01 k3s: time="2022-07-28T07:32:38.794077864+08:00" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/19bb6a9b46ad0013de084cf1e0feb7927ff9e4e06624685ff87f003c208fded1/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=rancher01 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --kubelet-cgroups=/systemd/system.slice --network-plugin=cni --node-labels= --pod-infra-container-image=docker.io/rancher/pause:3.1 --read-only-port=0 --resolv-conf=/etc/resolv.conf --runtime-cgroups=/systemd/system.slice --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Jul 28 07:32:38 rancher01 k3s: time="2022-07-28T07:32:38.815494478+08:00" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --healthz-bind-address=127.0.0.1 --hostname-override=rancher01 --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables"
Jul 28 07:32:38 rancher01 k3s: W0728 07:32:38.815625 63607 server.go:225] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.
Jul 28 07:32:38 rancher01 systemd: Started Kubernetes systemd probe.
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.845432 63607 server.go:413] Version: v1.18.8+k3s1
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.885047 63607 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.964860 63607 flannel.go:92] Determining IP address of default interface
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.973341 63607 flannel.go:105] Using interface with name eth0 and address 172.18.30.79
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.977423 63607 kube.go:117] Waiting 10m0s for node controller to sync
Jul 28 07:32:38 rancher01 k3s: I0728 07:32:38.977453 63607 kube.go:300] Starting kube subnet manager
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.152278 63607 node.go:136] Successfully retrieved node IP: 172.18.30.79
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.152327 63607 server_others.go:187] Using iptables Proxier.
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.178433 63607 server.go:583] Version: v1.18.8+k3s1
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.180107 63607 conntrack.go:52] Setting nf_conntrack_max to 131072
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.182527 63607 config.go:315] Starting service config controller
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.182543 63607 shared_informer.go:223] Waiting for caches to sync for service config
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.182563 63607 config.go:133] Starting endpoints config controller
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.182577 63607 shared_informer.go:223] Waiting for caches to sync for endpoints config
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213054 63607 server.go:645] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213637 63607 container_manager_linux.go:277] container manager verified user specified cgroup-root exists: []
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213656 63607 container_manager_linux.go:282] Creating Container Manager object based on Node Config: {RuntimeCgroupsName:/systemd/system.slice SystemCgroupsName: KubeletCgroupsName:/systemd/system.slice ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213780 63607 topology_manager.go:126] [topologymanager] Creating topology manager with none policy
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213795 63607 container_manager_linux.go:312] [topologymanager] Initializing Topology Manager with none policy
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213804 63607 container_manager_linux.go:317] Creating device plugin manager: true
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213884 63607 client.go:75] Connecting to docker on unix:///var/run/docker.sock
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.213905 63607 client.go:92] Start docker client with request timeout=2m0s
Jul 28 07:32:39 rancher01 k3s: time="2022-07-28T07:32:39.239358783+08:00" level=info msg="labels have already set on node: rancher01"
Jul 28 07:32:39 rancher01 k3s: W0728 07:32:39.285301 63607 docker_service.go:561] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.285343 63607 docker_service.go:238] Hairpin mode set to "hairpin-veth"
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.382586 63607 trace.go:116] Trace[1476740618]: "List etcd3" key:/clusterrolebindings,resourceVersion:,limit:0,continue: (started: 2022-07-28 07:32:36.559526541 +0800 CST m=+10.054248380) (total time: 2.823006322s):
Jul 28 07:32:39 rancher01 k3s: Trace[1476740618]: [2.823006322s] [2.823006322s] END
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.382787 63607 trace.go:116] Trace[161590881]: "List" url:/apis/rbac.authorization.k8s.io/v1/clusterrolebindings,user-agent:k3s/v1.18.8+k3s1 (linux/amd64) kubernetes/6b59531,client:127.0.0.1 (started: 2022-07-28 07:32:36.559491839 +0800 CST m=+10.054213660) (total time: 2.82326264s):
Jul 28 07:32:39 rancher01 k3s: Trace[161590881]: [2.823119071s] [2.823088969s] Listing from storage done
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.411452 63607 shared_informer.go:230] Caches are synced for service config
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.411519 63607 shared_informer.go:230] Caches are synced for endpoints config
Jul 28 07:32:39 rancher01 k3s: time="2022-07-28T07:32:39.430485688+08:00" level=error msg="error while range on /registry/clusterrolebindings/ /registry/clusterrolebindings/: rpc error: code = OutOfRange desc = etcdserver: mvcc: required revision has been compacted"
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.517712 63607 controller.go:606] quota admission added evaluator for: addons.k3s.cattle.io
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.528105 63607 docker_service.go:253] Docker cri networking managed by cni
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.586580 63607 docker_service.go:258] Docker Info: &{ID:LVSU:I4A3:YYGI:EZME:UPX5:BWLA:TSWZ:D6D7:KPQG:GCUG:5HF3:7A34 Containers:32 ContainersRunning:25 ContainersPaused:0 ContainersStopped:7 Images:111 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host macvlan null overlay] Authorization:[] Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:true Debug:false NFd:182 OomKillDisable:true NGoroutines:175 SystemTime:2022-07-28T07:32:39.530423353+08:00 LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.4.231-1.el7.elrepo.x86_64 OperatingSystem:CentOS Linux 7 (Core) OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc03d417180 NCPU:4 MemTotal:8352854016 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:rancher01 Labels:[] ExperimentalBuild:false ServerVersion:18.09.0 ClusterStore: ClusterAdvertise: Runtimes:map[runc:{Path:runc Args:[]}] DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:[] Nodes:0 Managers:0 Cluster:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:c4446665cb9c30056f4998ed953e6d4ff22c7c39 Expected:c4446665cb9c30056f4998ed953e6d4ff22c7c39} RuncCommit:{ID:4fc53a81fb7c994640722ac585fa9ca548971871 Expected:4fc53a81fb7c994640722ac585fa9ca548971871} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=seccomp,profile=default]}
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.586677 63607 docker_service.go:271] Setting cgroupDriver to cgroupfs
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.603947 63607 network_policy_controller.go:149] Starting network policy controller
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.643790 63607 kubelet.go:317] Watching apiserver
Jul 28 07:32:39 rancher01 k3s: time="2022-07-28T07:32:39.652895865+08:00" level=error msg="error while range on /registry/clusterrolebindings/ /registry/clusterrolebindings/: rpc error: code = OutOfRange desc = etcdserver: mvcc: required revision has been compacted"
Jul 28 07:32:39 rancher01 k3s: time="2022-07-28T07:32:39.848138420+08:00" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChart controller"
Jul 28 07:32:39 rancher01 k3s: time="2022-07-28T07:32:39.848539530+08:00" level=info msg="Starting batch/v1, Kind=Job controller"
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.852998 63607 kuberuntime_manager.go:211] Container runtime docker initialized, version: 18.09.0, apiVersion: 1.39.0
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.853375 63607 server.go:1124] Started kubelet
Jul 28 07:32:39 rancher01 k3s: E0728 07:32:39.870529 63607 kubelet.go:1306] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory cache
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.873859 63607 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.880157 63607 server.go:145] Starting to listen on 0.0.0.0:10250
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.881284 63607 server.go:393] Adding debug handlers to kubelet server.
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.901370 63607 volume_manager.go:265] Starting Kubelet Volume Manager
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.937422 63607 desired_state_of_world_populator.go:139] Desired state populator starts to run
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.993061 63607 kube.go:124] Node controller sync successful
Jul 28 07:32:39 rancher01 k3s: I0728 07:32:39.993120 63607 vxlan.go:121] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.006167 63607 kuberuntime_manager.go:978] updating runtime config through cri with podcidr 10.42.0.0/24
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.006472 63607 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.026330 63607 topology_manager.go:219] [topologymanager] RemoveContainer - Container ID: e3b0bea6fe213d5fe0fa333e4504c762a03734e40a2947b569a58909051f2035
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.026642 63607 docker_service.go:353] docker cri received runtime config &RuntimeConfig{NetworkConfig:&NetworkConfig{PodCidr:10.42.0.0/24,},}
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.071615 63607 kubelet_network.go:77] Setting Pod CIDR: -> 10.42.0.0/24
Jul 28 07:32:40 rancher01 k3s: time="2022-07-28T07:32:40.079658342+08:00" level=error msg="error while range on /registry/rolebindings/ /registry/rolebindings/: rpc error: code = OutOfRange desc = etcdserver: mvcc: required revision has been compacted"
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.088008 63607 trace.go:116] Trace[106727626]: "List etcd3" key:/management.cattle.io/catalogs,resourceVersion:,limit:10000,continue: (started: 2022-07-28 07:32:39.269014036 +0800 CST m=+12.763735880) (total time: 818.969247ms):
Jul 28 07:32:40 rancher01 k3s: Trace[106727626]: [818.969247ms] [818.969247ms] END
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.088156 63607 trace.go:116] Trace[843184200]: "cacher list" type:*unstructured.Unstructured (started: 2022-07-28 07:32:39.256041833 +0800 CST m=+12.750763770) (total time: 832.099178ms):
Jul 28 07:32:40 rancher01 k3s: Trace[843184200]: [832.086164ms] [832.086164ms] Ready
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.088756 63607 trace.go:116] Trace[1727399713]: "List" url:/apis/management.cattle.io/v3/catalogs,user-agent:Go-http-client/1.1,client:10.42.0.88 (started: 2022-07-28 07:32:39.255998782 +0800 CST m=+12.750720601) (total time: 832.729434ms):
Jul 28 07:32:40 rancher01 k3s: Trace[1727399713]: [832.207264ms] [832.171918ms] Listing from storage done
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.130639 63607 trace.go:116] Trace[855850943]: "GuaranteedUpdate etcd3" type:*unstructured.Unstructured (started: 2022-07-28 07:32:39.517133885 +0800 CST m=+13.011855744) (total time: 613.481456ms):
Jul 28 07:32:40 rancher01 k3s: Trace[855850943]: [613.481456ms] [613.40311ms] END
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.130826 63607 trace.go:116] Trace[1528878223]: "Update" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons/ccm,user-agent:k3s/v1.18.8+k3s1 (linux/amd64) kubernetes/6b59531,client:127.0.0.1 (started: 2022-07-28 07:32:39.516672775 +0800 CST m=+13.011394613) (total time: 614.131241ms):
Jul 28 07:32:40 rancher01 k3s: Trace[1528878223]: [614.01547ms] [613.602912ms] Object stored in database
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.134651 63607 controller.go:606] quota admission added evaluator for: helmcharts.helm.cattle.io
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.170813 63607 trace.go:116] Trace[1840478679]: "List etcd3" key:/rolebindings,resourceVersion:268891438,limit:500,continue: (started: 2022-07-28 07:32:38.697519277 +0800 CST m=+12.192241098) (total time: 1.473247947s):
Jul 28 07:32:40 rancher01 k3s: Trace[1840478679]: [1.473247947s] [1.473247947s] END
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.170898 63607 trace.go:116] Trace[386931616]: "List" url:/apis/rbac.authorization.k8s.io/v1/rolebindings,user-agent:Go-http-client/1.1,client:10.42.0.88 (started: 2022-07-28 07:32:38.697480755 +0800 CST m=+12.192202593) (total time: 1.47339301s):
Jul 28 07:32:40 rancher01 k3s: Trace[386931616]: [1.47339301s] [1.473362509s] END
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.475811 63607 trace.go:116] Trace[1353523385]: "List etcd3" key:/clusterrolebindings,resourceVersion:,limit:500,continue: (started: 2022-07-28 07:32:39.711180253 +0800 CST m=+13.205902090) (total time: 764.602468ms):
Jul 28 07:32:40 rancher01 k3s: Trace[1353523385]: [764.602468ms] [764.602468ms] END
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.490444 63607 status_manager.go:158] Starting to sync pod status with apiserver
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.490478 63607 kubelet.go:1822] Starting kubelet main sync loop.
Jul 28 07:32:40 rancher01 k3s: E0728 07:32:40.490530 63607 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.505169 63607 trace.go:116] Trace[1054428592]: "List" url:/apis/rbac.authorization.k8s.io/v1/clusterrolebindings,user-agent:rancher/v0.0.0 (linux/amd64) kubernetes/$Format,client:10.42.0.88 (started: 2022-07-28 07:32:39.711154466 +0800 CST m=+13.205876489) (total time: 793.977529ms):
Jul 28 07:32:40 rancher01 k3s: Trace[1054428592]: [764.677266ms] [764.656999ms] Listing from storage done
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.517484 63607 trace.go:116] Trace[246212888]: "List etcd3" key:/clusterrolebindings,resourceVersion:,limit:500,continue:eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6MjY4OTUyMjc4LCJzdGFydCI6ImNsdXN0ZXJyb2xlYmluZGluZy1keGcyNFx1MDAwMCJ9 (started: 2022-07-28 07:32:39.837555671 +0800 CST m=+13.332277563) (total time: 679.898969ms):
Jul 28 07:32:40 rancher01 k3s: Trace[246212888]: [679.898969ms] [679.898969ms] END
Jul 28 07:32:40 rancher01 k3s: E0728 07:32:40.590820 63607 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.603687 63607 trace.go:116] Trace[713667402]: "List" url:/apis/rbac.authorization.k8s.io/v1/clusterrolebindings,user-agent:Go-http-client/1.1,client:10.42.0.88 (started: 2022-07-28 07:32:39.837531213 +0800 CST m=+13.332253114) (total time: 766.112503ms):
Jul 28 07:32:40 rancher01 k3s: Trace[713667402]: [679.970469ms] [679.952734ms] Listing from storage done
Jul 28 07:32:40 rancher01 k3s: I0728 07:32:40.607206 63607 trace.go:116] Trace[1310988190]: "List etcd3" key:/clusterrolebindings,resourceVersion:,limit:500,continue:eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6MjY4OTUyMjczLCJzdGFydCI6ImNsdXN0ZXJyb2xlYmluZGluZy1xZzJ0Zlx1MDAwMCJ9 (started: 2022-07-28 07:32:39.585565076 +0800 CST m=+13.080286898) (total time: 1.021411901s):