helm安装Rancher后访问404

Rancher Server 设置

  • Rancher 版本:2.6
  • 安装选项 (Docker install/Helm Chart): Helm Chart安装
    • 如果是 Helm Chart 安装,需要提供 Local 集群的类型(RKE1, RKE2, k3s, EKS, 等)和版本:
  • 在线或离线部署:

下游集群信息

  • Kubernetes 版本: 1.23
  • Cluster Type (Local/Downstream):
    • 如果 Downstream,是什么类型的集群?(自定义/导入或为托管 等):

用户信息

  • 登录用户的角色是什么? (管理员/集群所有者/集群成员/项目所有者/项目成员/自定义):
    • 如果自定义,自定义权限集:

问题描述:
helm安装,安装后状态都是running,但是域名访问404

重现步骤:
1.Install the Rancher Helm Chart,version:v3.9.0

  1. Add the Helm Chart Repository
    helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
    kubectl create namespace cattle-system

3.Adding TLS Secrets
kubectl -n cattle-system create secret tls tls-rancher-ingress
–cert=tls.crt
–key=tls.key

  1. Install Rancher with Helm and Your Chosen Certificate Option

helm install rancher rancher-stable/rancher
–namespace cattle-system
–set hostname=xxx.com.cn
–set bootstrapPassword=admin
–set ingress.tls.source=secret

  1. Verify that the Rancher Server,rancher版本(V2.6.5)

kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment “rancher” rollout to finish: 0 of 3 updated replicas are available…

6.確認啟動均正常
[root@master deployment]# kubectl get pod -n cattle-system
NAME READY STATUS RESTARTS AGE
helm-operation-gnf5r 0/2 Completed 0 23m
helm-operation-jkj8l 0/2 Completed 0 24m
helm-operation-vft8r 0/2 Completed 0 23m
rancher-7bbd98588-5b7gp 1/1 Running 0 44m
rancher-7bbd98588-r6rbv 1/1 Running 0 44m
rancher-7bbd98588-z5kxn 1/1 Running 0 44m
rancher-webhook-5b65595df9-lczfk 1/1 Running 0 22m

7.網頁訪問測試,400報錯
[root@master deployment]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 172.110.115.194 80:31825/TCP,443:31887/TCP 59d
ingress-nginx-controller-admission ClusterIP 172.110.238.77 443/TCP 59d

訪問:http://k8s-master.flexium.com.cn:31887/
訪問狀態:404

pod日誌:
2022/06/29 08:32:59 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-local-system, err=Operation cannot be fulfilled on namespaces “cattle-fleet-local-system”: the object has been modified; please apply your changes to the latest version and try again
2022/06/29 08:33:00 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-local-system, err=Operation cannot be fulfilled on namespaces “cattle-fleet-local-system”: the object has been modified; please apply your changes to the latest version and try again

Ingress 对象的状态
[root@master ~]# kubectl -n cattle-system describe ingress
Name: rancher
Labels: app=rancher
app.kubernetes.io/managed-by=Helm
chart=rancher-2.6.5
heritage=Helm
release=rancher
Namespace: cattle-system
Address: 10.2.83.183
Default backend: default-http-backend:80 (<error: endpoints “default-http-backend” not found>)
TLS:
tls-rancher-ingress terminates k8s-master.flexium.com.cn
Rules:
Host Path Backends


k8s-master.flexium.com.cn
rancher:80 (172.100.219.106:80,172.100.247.3:80,172.100.84.187:80)
Annotations: field.cattle.io/publicEndpoints:
[{“addresses”:[“10.2.83.183”],“port”:443,“protocol”:“HTTPS”,“serviceName”:“cattle-system:rancher”,“ingressName”:“cattle-system:rancher”,"h…
meta.helm.sh/release-name: rancher
meta.helm.sh/release-namespace: cattle-system
nginx.ingress.kubernetes.io/proxy-connect-timeout: 30
nginx.ingress.kubernetes.io/proxy-read-timeout: 1800
nginx.ingress.kubernetes.io/proxy-send-timeout: 1800
Events:

结果:
访问404
预期结果:

截图:
image

其他上下文信息:

日志
pod日誌:
2022/06/29 08:32:59 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-local-system, err=Operation cannot be fulfilled on namespaces "cattle-fleet-local-system": the object has been modified; please apply your changes to the latest version and try again
2022/06/29 08:33:00 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-local-system, err=Operation cannot be fulfilled on namespaces "cattle-fleet-local-system": the object has been modified; please apply your changes to the latest version and try again


Ingress 对象的状态
[root@master ~]# kubectl -n cattle-system describe ingress
Name:             rancher
Labels:           app=rancher
                  app.kubernetes.io/managed-by=Helm
                  chart=rancher-2.6.5
                  heritage=Helm
                  release=rancher
Namespace:        cattle-system
Address:          10.2.83.183
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  tls-rancher-ingress terminates k8s-master.flexium.com.cn
Rules:
  Host                       Path  Backends
  ----                       ----  --------
  k8s-master.flexium.com.cn  
                                rancher:80 (172.100.219.106:80,172.100.247.3:80,172.100.84.187:80)
Annotations:                 field.cattle.io/publicEndpoints:
                               [{"addresses":["10.2.83.183"],"port":443,"protocol":"HTTPS","serviceName":"cattle-system:rancher","ingressName":"cattle-system:rancher","h...
                             meta.helm.sh/release-name: rancher
                             meta.helm.sh/release-namespace: cattle-system
                             nginx.ingress.kubernetes.io/proxy-connect-timeout: 30
                             nginx.ingress.kubernetes.io/proxy-read-timeout: 1800
                             nginx.ingress.kubernetes.io/proxy-send-timeout: 1800
Events:                      <none>


你的 tls 证书是怎么生成的?

公司购买的泛域名,我用的公司的证书

先按照这个排查下吧,Rancher使用NGINX进行反向代理无效 - #2,来自 ksd

你好,我有排查,我的ingress启动和rancher pod启动都是正常的,除了pod有此warns:
policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+

其次通过查看ingress描述,有此报错,我不知道是否相关
kubectl -n cattle-system describe ingress

Name: rancher
Labels: app=rancher
app.kubernetes.io/managed-by=Helm
chart=rancher-2.6.5
heritage=Helm
release=rancher
Namespace: cattle-system
Address: 10.2.83.183
Default backend: default-http-backend:80 (<error: endpoints “default-http-backend” not found>)
TLS:

你如果使用Ingress做TLS Terminate,就应该用Ingress的入口访问。如果你安装的nginx-ingress,一般都是80/443访问入口。
不过,你提供404访问url,看起来是一个nodeport端口?

你的Local集群是什么类型,ingress-controller 如何部署?

你好,我的K8S集群,就是kubeadm init安装的,我这个helm安装完rancher,还需要配置ingress-nginx吗

你的kubeadm的安装方式,我无从得知。但是,你如果使用ingress做TLS terminate,有一个ingress-controller是必须,它可以是ingress-nginx,也可以是traefik之类的。

Rancher在官方文档的quick-start中,使用k3s,因为k3s内置了traefik ingress:Rancher Docs: Helm CLI Quick Start
不过,quick-start不适用于生产环境。但是,部署原理是相同的。

Rancher会默认测试以K3s/RKE/RKE2作为Local集群的安装质量:Rancher Docs: 3. Install Kubernetes (Skip for Docker Installs)

然而Kubeadm则不是测试的选项,因为用户普遍会自定义kubeadm,无法判断用户自身的部署质量。