Rke2禁用rke2-ingress-nginx后如何恢复

环境信息:
RKE2 版本: 1.26.12

节点 CPU 架构,操作系统和版本:
Linux eu-server7 3.10.0-1160.102.1.0.1.an7.x86_64 #1 SMP Sun Oct 29 06:40:18 CST 2023 x86_64 x86_64 x86_64 GNU/Linux

集群配置:
“3 servers, 1 agents”.

问题描述:
rke2集群禁用自带的rke2-ingress-nginx后,想恢复。发现恢复不了,请大神帮看看怎么解决,谢谢。

helmchart如下

/etc/rancher/rke2/config.yaml

...

#disable: rke2-ingress-nginx

你得把所有节点的的 /etc/rancher/rke2/config.yaml 中的 ingress 配置都去掉,然后重启 rke2 服务试试。

如果还不行,我在想办法重现试试

是所有server节点都去掉disable ingress配置了,并且重启服务器也不行。
我是因为配置ingress过程中发现ingress不工作了不监听80,443端口,所以禁用了nginx-ingress。
看来这种禁用又启用的方法不会重置rke2-ingress-nginx。请教下有什么重新安装rke2-ingress-nginx的方法么

我使用相同的架构和版本去测试,没重现,我还是列下我的操作步骤吧,你看看是否有啥不同:

环境信息

  • server1:ip-172-31-15-8
  • server2:ip-172-31-11-230
  • server3:ip-172-31-11-74

rke2 版本:v1.26.12+rke2r1

OS: Ubuntu 20.04.6 LTS

启动 RKE2 3 server 集群

  1. 启动第一个 rke2 server
root@ip-172-31-15-8:~# cat /etc/rancher/rke2/config.yaml
token: token

root@ip-172-31-15-8:~# curl -sfL https://get.rke2.io | sh -
root@ip-172-31-15-8:~# systemctl enable rke2-server.service
root@ip-172-31-15-8:~# systemctl start rke2-server.service
  1. 启动另外两个 server
root@ip-172-31-11-230:~# cat /etc/rancher/rke2/config.yaml
server: https://172.31.15.8:9345
token: token

root@ip-172-31-11-230:~# systemctl enable rke2-server.service
root@ip-172-31-11-230:~# systemctl enable rke2-server.service
root@ip-172-31-11-230:~# systemctl start rke2-server.service

第三个 server 的步骤省略,和 server 2 一样

  1. 查看 节点状态
root@ip-172-31-15-8:~# /var/lib/rancher/rke2/bin/kubectl get nodes
NAME               STATUS   ROLES                       AGE   VERSION
ip-172-31-11-230   Ready    control-plane,etcd,master   30m   v1.26.12+rke2r1
ip-172-31-11-74    Ready    control-plane,etcd,master   21m   v1.26.12+rke2r1
ip-172-31-15-8     Ready    control-plane,etcd,master   33m   v1.26.12+rke2r1
  1. 查看 pod 状态
root@ip-172-31-15-8:~# /var/lib/rancher/rke2/bin/kubectl get pods -A
NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-ip-172-31-11-230               1/1     Running     0          30m
kube-system   cloud-controller-manager-ip-172-31-11-74                1/1     Running     0          21m
kube-system   cloud-controller-manager-ip-172-31-15-8                 1/1     Running     0          33m
kube-system   etcd-ip-172-31-11-230                                   1/1     Running     0          30m
kube-system   etcd-ip-172-31-11-74                                    1/1     Running     0          21m
kube-system   etcd-ip-172-31-15-8                                     1/1     Running     0          33m
kube-system   helm-install-rke2-canal-rzf7p                           0/1     Completed   0          33m
kube-system   helm-install-rke2-coredns-g2v5c                         0/1     Completed   0          33m
kube-system   helm-install-rke2-ingress-nginx-b6n6r                   0/1     Completed   0          33m
kube-system   helm-install-rke2-metrics-server-hmpqm                  0/1     Completed   0          33m
kube-system   helm-install-rke2-snapshot-controller-crd-jf6g4         0/1     Completed   0          33m
kube-system   helm-install-rke2-snapshot-controller-zrb9c             0/1     Completed   0          33m
kube-system   helm-install-rke2-snapshot-validation-webhook-ccxzb     0/1     Completed   0          33m
kube-system   kube-apiserver-ip-172-31-11-230                         1/1     Running     0          30m
kube-system   kube-apiserver-ip-172-31-11-74                          1/1     Running     0          21m
kube-system   kube-apiserver-ip-172-31-15-8                           1/1     Running     0          33m
kube-system   kube-controller-manager-ip-172-31-11-230                1/1     Running     0          30m
kube-system   kube-controller-manager-ip-172-31-11-74                 1/1     Running     0          21m
kube-system   kube-controller-manager-ip-172-31-15-8                  1/1     Running     0          33m
kube-system   kube-proxy-ip-172-31-11-230                             1/1     Running     0          30m
kube-system   kube-proxy-ip-172-31-11-74                              1/1     Running     0          21m
kube-system   kube-proxy-ip-172-31-15-8                               1/1     Running     0          33m
kube-system   kube-scheduler-ip-172-31-11-230                         1/1     Running     0          30m
kube-system   kube-scheduler-ip-172-31-11-74                          1/1     Running     0          21m
kube-system   kube-scheduler-ip-172-31-15-8                           1/1     Running     0          33m
kube-system   rke2-canal-q4x6j                                        2/2     Running     0          30m
kube-system   rke2-canal-vrhtt                                        2/2     Running     0          22m
kube-system   rke2-canal-zmkhq                                        2/2     Running     0          33m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-nz5m7              1/1     Running     0          33m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-td8zb              1/1     Running     0          30m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-c9dcs   1/1     Running     0          33m
kube-system   rke2-ingress-nginx-controller-78kzw                     1/1     Running     0          30m
kube-system   rke2-ingress-nginx-controller-pc2x7                     1/1     Running     0          21m
kube-system   rke2-ingress-nginx-controller-qghgt                     1/1     Running     0          32m
kube-system   rke2-metrics-server-c9c78bd66-8z4h7                     1/1     Running     0          33m
kube-system   rke2-snapshot-controller-6f7bbb497d-jjgsb               1/1     Running     0          32m
kube-system   rke2-snapshot-validation-webhook-65b5675d5c-vdmgh       1/1     Running     0          32m

以上,可以看出自动部署了 rke2-ingress-nginx

禁用 rke2-ingress-nginx

三个server 节点分别添加 disable: rke2-ingress-nginx 参数

server1:

root@ip-172-31-15-8:~# cat /etc/rancher/rke2/config.yaml
token: token
disable: rke2-ingress-nginx

server2:

root@ip-172-31-11-230:~# cat /etc/rancher/rke2/config.yaml
server: https://172.31.15.8:9345
token: token
disable: rke2-ingress-nginx

server3:

root@ip-172-31-11-74:~# cat /etc/rancher/rke2/config.yaml
server: https://172.31.15.8:9345
token: token
disable: rke2-ingress-nginx

三个 server 节点分别重启:

systemctl restart rke2-server.service

查看pod 状态:

NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-ip-172-31-11-230               1/1     Running     0          33m
kube-system   cloud-controller-manager-ip-172-31-11-74                1/1     Running     0          24m
kube-system   cloud-controller-manager-ip-172-31-15-8                 1/1     Running     0          36m
kube-system   etcd-ip-172-31-11-230                                   1/1     Running     0          33m
kube-system   etcd-ip-172-31-11-74                                    1/1     Running     0          24m
kube-system   etcd-ip-172-31-15-8                                     1/1     Running     0          36m
kube-system   helm-delete-rke2-ingress-nginx-cdm65                    0/1     Completed   0          26s
kube-system   helm-install-rke2-canal-rzf7p                           0/1     Completed   0          36m
kube-system   helm-install-rke2-coredns-g2v5c                         0/1     Completed   0          36m
kube-system   helm-install-rke2-metrics-server-hmpqm                  0/1     Completed   0          36m
kube-system   helm-install-rke2-snapshot-controller-crd-jf6g4         0/1     Completed   0          36m
kube-system   helm-install-rke2-snapshot-controller-zrb9c             0/1     Completed   0          36m
kube-system   helm-install-rke2-snapshot-validation-webhook-ccxzb     0/1     Completed   0          36m
kube-system   kube-apiserver-ip-172-31-11-230                         1/1     Running     0          33m
kube-system   kube-apiserver-ip-172-31-11-74                          1/1     Running     0          24m
kube-system   kube-apiserver-ip-172-31-15-8                           1/1     Running     0          36m
kube-system   kube-controller-manager-ip-172-31-11-230                1/1     Running     0          33m
kube-system   kube-controller-manager-ip-172-31-11-74                 1/1     Running     0          24m
kube-system   kube-controller-manager-ip-172-31-15-8                  1/1     Running     0          36m
kube-system   kube-proxy-ip-172-31-11-230                             1/1     Running     0          33m
kube-system   kube-proxy-ip-172-31-11-74                              1/1     Running     0          24m
kube-system   kube-proxy-ip-172-31-15-8                               1/1     Running     0          36m
kube-system   kube-scheduler-ip-172-31-11-230                         1/1     Running     0          33m
kube-system   kube-scheduler-ip-172-31-11-74                          1/1     Running     0          24m
kube-system   kube-scheduler-ip-172-31-15-8                           1/1     Running     0          36m
kube-system   rke2-canal-q4x6j                                        2/2     Running     0          33m
kube-system   rke2-canal-vrhtt                                        2/2     Running     0          25m
kube-system   rke2-canal-zmkhq                                        2/2     Running     0          36m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-nz5m7              1/1     Running     0          36m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-td8zb              1/1     Running     0          33m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-c9dcs   1/1     Running     0          36m
kube-system   rke2-metrics-server-c9c78bd66-8z4h7                     1/1     Running     0          36m
kube-system   rke2-snapshot-controller-6f7bbb497d-jjgsb               1/1     Running     0          35m
kube-system   rke2-snapshot-validation-webhook-65b5675d5c-vdmgh       1/1     Running     0          35m

以上可以看到,rke2-ingress-nginx 已经通过 helm-delete-rke2-ingress-nginx job 删掉了

再次启用 rke2-ingress-nginx

三个server 节点分别 去掉 disable: rke2-ingress-nginx 参数

server1:

root@ip-172-31-15-8:~# cat /etc/rancher/rke2/config.yaml
token: token
#disable: rke2-ingress-nginx

server2:

root@ip-172-31-11-230:~# cat /etc/rancher/rke2/config.yaml
server: https://172.31.15.8:9345
token: token
#disable: rke2-ingress-nginx

server3:

root@ip-172-31-11-74:~# cat /etc/rancher/rke2/config.yaml
server: https://172.31.15.8:9345
token: token
#disable: rke2-ingress-nginx

三个 server 节点分别重启:

systemctl restart rke2-server.service

查看pod 状态:

root@ip-172-31-15-8:~# /var/lib/rancher/rke2/bin/kubectl get pods -A
NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-ip-172-31-11-230               1/1     Running     0          48m
kube-system   cloud-controller-manager-ip-172-31-11-74                1/1     Running     0          39m
kube-system   cloud-controller-manager-ip-172-31-15-8                 1/1     Running     0          51m
kube-system   etcd-ip-172-31-11-230                                   1/1     Running     0          48m
kube-system   etcd-ip-172-31-11-74                                    1/1     Running     0          39m
kube-system   etcd-ip-172-31-15-8                                     1/1     Running     0          51m
kube-system   helm-delete-rke2-ingress-nginx-cdm65                    0/1     Completed   0          15m
kube-system   helm-install-rke2-canal-rzf7p                           0/1     Completed   0          51m
kube-system   helm-install-rke2-coredns-g2v5c                         0/1     Completed   0          51m
kube-system   helm-install-rke2-ingress-nginx-ssffb                   0/1     Completed   0          14m
kube-system   helm-install-rke2-metrics-server-hmpqm                  0/1     Completed   0          51m
kube-system   helm-install-rke2-snapshot-controller-crd-jf6g4         0/1     Completed   0          51m
kube-system   helm-install-rke2-snapshot-controller-zrb9c             0/1     Completed   0          51m
kube-system   helm-install-rke2-snapshot-validation-webhook-ccxzb     0/1     Completed   0          51m
kube-system   kube-apiserver-ip-172-31-11-230                         1/1     Running     0          48m
kube-system   kube-apiserver-ip-172-31-11-74                          1/1     Running     0          39m
kube-system   kube-apiserver-ip-172-31-15-8                           1/1     Running     0          51m
kube-system   kube-controller-manager-ip-172-31-11-230                1/1     Running     0          48m
kube-system   kube-controller-manager-ip-172-31-11-74                 1/1     Running     0          39m
kube-system   kube-controller-manager-ip-172-31-15-8                  1/1     Running     0          51m
kube-system   kube-proxy-ip-172-31-11-230                             1/1     Running     0          48m
kube-system   kube-proxy-ip-172-31-11-74                              1/1     Running     0          39m
kube-system   kube-proxy-ip-172-31-15-8                               1/1     Running     0          51m
kube-system   kube-scheduler-ip-172-31-11-230                         1/1     Running     0          48m
kube-system   kube-scheduler-ip-172-31-11-74                          1/1     Running     0          39m
kube-system   kube-scheduler-ip-172-31-15-8                           1/1     Running     0          51m
kube-system   rke2-canal-q4x6j                                        2/2     Running     0          48m
kube-system   rke2-canal-vrhtt                                        2/2     Running     0          40m
kube-system   rke2-canal-zmkhq                                        2/2     Running     0          51m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-nz5m7              1/1     Running     0          51m
kube-system   rke2-coredns-rke2-coredns-565dfc7d75-td8zb              1/1     Running     0          48m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-c9dcs   1/1     Running     0          51m
kube-system   rke2-ingress-nginx-controller-7t9m4                     1/1     Running     0          13m
kube-system   rke2-ingress-nginx-controller-bwq9k                     1/1     Running     0          13m
kube-system   rke2-ingress-nginx-controller-lq4dz                     1/1     Running     0          13m
kube-system   rke2-metrics-server-c9c78bd66-8z4h7                     1/1     Running     0          51m
kube-system   rke2-snapshot-controller-6f7bbb497d-jjgsb               1/1     Running     0          50m
kube-system   rke2-snapshot-validation-webhook-65b5675d5c-vdmgh       1/1     Running     0          50m

rke2-ingress-nginx 再次通过 job 添加回来了,以上可以看出,禁用或启动 rke2-ingress-nginx,可通过 参数 disable: rke2-ingress-nginx 即可控制。