环境信息:
K3s 版本: v1.26.3+k3s1
节点 CPU 架构、操作系统和版本：: x86_64/CentOS Linux release 7.9.2009 (Core)/3.10.0-1160.el7.x86_64
k3s version v1.26.3+k3s1 (01ea3ff2)
go version go1.19.7
集群配置:
2 master，外部数据库
问题描述:
测试yaml如下：
apiVersion: apps/v1
kind: Deployment
metadata:
name: test1
spec:
replicas: 3
selector:
matchLabels:
app: test1
template:
metadata:
labels:
app: test1
spec:
containers:
- name: chenby
image: nginx
imagePullPolicy: IfNotPresent
resources:
limits:
memory: “256Mi”
cpu: “500m”
ports:
- containerPort: 80
apiVersion: v1
kind: Service
metadata:
name: test1
spec:
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv6
- IPv4
type: NodePort
selector:
app: test1
ports: - port: 80
targetPort: 80
SVC信息如下:
[root@hexin1 fant]# kubectl get svc -A
default test1 NodePort fd43::3cc5 80:60355/TCP 5m13s
EP信息如下：
[root@hexin1 fant]# kubectl get ep
test1 [fd42::32af:958:4f65:ee68:a4c7]:80,[fd42::c98c:4145:7d7d:5104:3a01]:80,[fd42::c98c:4145:7d7d:5104:3a02]:80 5m52s
问题如下：
[root@hexin2 ~]# curl -g -I http://[fd43::3cc5]
在使用curl 请求的时候，请求轮询到本节点上的pod是可以返回的，但轮询访问到1节点上的pods是会夯住，最后会返回连接超时，如下：
[root@hexin2 ~]# curl -g -I http://[fd43::3cc5]
curl: (7) Failed connect to fd43::3cc5:80; 没有到主机的路由
系统sysctl.conf配置如下：
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness = 0
vm.panic_on_oom = 0
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.all.forwarding = 1
复现步骤:
- 安装 K3s 的命令:
/usr/local/bin/k3s
server
‘–datastore-endpoint=postgres://pgmonitor:pgmonitor@10.0.0.204:20001/k3s?sslmode=disable’
‘–kube-apiserver-arg’
‘service-node-port-range=1-65000’
‘–disable’
‘traefik’
‘–disable’
‘metrics-server’
‘–disable’
‘servicelb’
‘–disable-cloud-controller’
‘–kube-proxy-arg’
‘proxy-mode=ipvs’
‘–cluster-cidr=10.42.0.0/16,fd42::/48’
‘–service-cidr=10.43.0.0/16,fd43::/112’
‘–disable-network-policy’
‘–flannel-backend=none’
‘–node-ip=10.0.0.11,2023::10:0:0:11’
但查看ipvs规则是这样的：
[root@hexin2 tmp]# ipvsadm -Ln --stats
TCP [fd43::3cc5]:80 10 44 26 3742 3460
→ [fd42::32af:958:4f65:ee68:a4c7]:80 3 3 2 240 256
→ [fd42::c98c:4145:7d7d:5104:3a01]:80 3 18 12 1551 1602
→ [fd42::c98c:4145:7d7d:5104:3a02]:80 3 18 12 1551 1602
TCP [2023::10:0:0:11]:60355 0 0 0 0 0
→ [fd42::32af:958:4f65:ee68:a4c7]:80 0 0 0 0 0
→ [fd42::c98c:4145:7d7d:5104:3a01]:80 0 0 0 0 0
→ [fd42::c98c:4145:7d7d:5104:3a02]:80 0 0 0 0 0