K3s参照国内安装手册安装后flannel网络异常

lbzss · 2024 年10 月 25 日 01:33

环境信息:
K3s 版本:
k3s version v1.30.5+k3s1 (9b586704)

节点 CPU 架构、操作系统和版本：:
Linux k8s0001 5.4.278-1.el7.elrepo.x86_64 #1 SMP Sun Jun 16 15:37:11 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux

集群配置:
3 servers, 2 agents

问题描述:

参照教程使用国内资源安装 K3s 全攻略 - 权威教程 - Rancher 中文论坛安装K3s集群后，跨节点pod无法ping通，各节点的flannel.1也不通，抓包看是flannel.1到本机的eth0不通
复现步骤:

安装 K3s 的命令:
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh |
INSTALL_K3S_MIRROR=cn
K3S_TOKEN=12345
sh -s - server
–cluster-init
–token 12345
–system-default-registry=registry.cn-hangzhou.aliyuncs.com

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh |
INSTALL_K3S_MIRROR=cn
K3S_TOKEN=12345
sh -s - server
–server https://10.89.64.72:6443
–system-default-registry=registry.cn-hangzhou.aliyuncs.com

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh |
INSTALL_K3S_MIRROR=cn
K3S_URL=https://10.89.64.72:6443
K3S_TOKEN=12345
sh -

预期结果:
跨节点网络可以ping通

实际结果:
跨节点pod不通

附加上下文/日志:

Node1

[root@k8s0001 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether fe:fc:fe:83:86:38 brd ff:ff:ff:ff:ff:ff
inet 10.89.64.72/24 brd 10.89.64.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::fcfc:feff:fe83:8638/64 scope link
valid_lft forever preferred_lft forever
3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether 06:82:fc:55:09:fa brd ff:ff:ff:ff:ff:ff
4: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether ce:45:a0:fd:93:59 brd ff:ff:ff:ff:ff:ff
inet 10.42.0.1/24 brd 10.42.0.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::cc45:a0ff:fefd:9359/64 scope link
valid_lft forever preferred_lft forever
5: vethbab2f694@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether ce:6a:2d:f7:73:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::cc6a:2dff:fef7:73e5/64 scope link
valid_lft forever preferred_lft forever
6: veth18d50cdc@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 1a:e7:ae:90:1d:75 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::18e7:aeff:fe90:1d75/64 scope link
valid_lft forever preferred_lft forever
7: vethc483d5c6@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 3a:60:1d:a1:50:ad brd ff:ff:ff:ff:ff:ff link-netnsid 2
inet6 fe80::3860:1dff:fea1:50ad/64 scope link
valid_lft forever preferred_lft forever
[root@k8s0001 ~]# ip route
default via 10.89.64.1 dev eth0 metric 103
10.42.0.0/24 dev cni0 proto kernel scope link src 10.42.0.1
10.42.1.0/24 via 10.42.1.0 dev flannel.1 onlink
10.42.2.0/24 via 10.42.2.0 dev flannel.1 onlink
10.42.3.0/24 via 10.42.3.0 dev flannel.1 onlink
10.42.4.0/24 via 10.42.4.0 dev flannel.1 onlink
10.89.64.0/24 dev eth0 proto kernel scope link src 10.89.64.72
169.254.0.0/16 dev eth0 scope link metric 1002
[root@k8s0001 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 16324 packets, 2782K bytes)
pkts bytes target prot opt in out source destination
22378 8940K KUBE-ROUTER-INPUT all – * * 0.0.0.0/0 0.0.0.0/0 /* kube-router netpol - 4IA2OSFRMVNDXBVV /
428 82337 KUBE-PROXY-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes load balancer firewall /
21012 8648K KUBE-NODEPORTS all – * * 0.0.0.0/0 0.0.0.0/0 / kubernetes health check service ports /
428 82337 KUBE-EXTERNAL-SERVICES all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes externally-visible service portals /
21012 8648K KUBE-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0
1 60 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 / KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies */ mark match 0x20000/0x20000

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
357 90244 KUBE-ROUTER-FORWARD all – * * 0.0.0.0/0 0.0.0.0/0 /* kube-router netpol - TEMCG2JMHZYE7H7T /
0 0 KUBE-PROXY-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes load balancer firewall /
0 0 KUBE-FORWARD all – * * 0.0.0.0/0 0.0.0.0/0 / kubernetes forwarding rules /
0 0 KUBE-SERVICES all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes service portals /
0 0 KUBE-EXTERNAL-SERVICES all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes externally-visible service portals /
0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 / KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies / mark match 0x20000/0x20000
0 0 ACCEPT all – * * 10.42.0.0/16 0.0.0.0/0
0 0 ACCEPT all – * * 0.0.0.0/0 10.42.0.0/16
0 0 FLANNEL-FWD all – * * 0.0.0.0/0 0.0.0.0/0 / flanneld forward */

Chain OUTPUT (policy ACCEPT 16088 packets, 2761K bytes)
pkts bytes target prot opt in out source destination
22165 8715K KUBE-ROUTER-OUTPUT all – * * 0.0.0.0/0 0.0.0.0/0 /* kube-router netpol - VEAAIY32XVBHCSCY /
76 4772 KUBE-PROXY-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes load balancer firewall /
76 4772 KUBE-SERVICES all – * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW / kubernetes service portals /
20747 8591K KUBE-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 / KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies */ mark match 0x20000/0x20000

Chain FLANNEL-FWD (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all – * * 10.42.0.0/16 0.0.0.0/0 /* flanneld forward /
0 0 ACCEPT all – * * 0.0.0.0/0 10.42.0.0/16 / flanneld forward */

Chain KUBE-EXTERNAL-SERVICES (2 references)
pkts bytes target prot opt in out source destination

Chain KUBE-FIREWALL (2 references)
pkts bytes target prot opt in out source destination
0 0 DROP all – * * !127.0.0.0/8 127.0.0.0/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-FORWARD (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all – * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules / mark match 0x4000/0x4000
0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 / kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-KUBELET-CANARY (0 references)
pkts bytes target prot opt in out source destination

Chain KUBE-NODEPORTS (1 references)
pkts bytes target prot opt in out source destination

Chain KUBE-NWPLCY-DEFAULT (6 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT icmp – * * 0.0.0.0/0 0.0.0.0/0 /* allow icmp echo requests / icmptype 8
0 0 ACCEPT icmp – * * 0.0.0.0/0 0.0.0.0/0 / allow icmp destination unreachable messages / icmptype 3
0 0 ACCEPT icmp – * * 0.0.0.0/0 0.0.0.0/0 / allow icmp time exceeded messages / icmptype 11
1 60 MARK all – * * 0.0.0.0/0 0.0.0.0/0 / rule to mark traffic matching a network policy */ MARK or 0x10000

Chain KUBE-POD-FW-4KKVF6GINJH2NPO6 (7 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 /* rule for stateful firewall for pod / ctstate RELATED,ESTABLISHED
0 0 DROP all – * * 0.0.0.0/0 0.0.0.0/0 / rule to drop invalid state for pod / ctstate INVALID
0 0 ACCEPT all – * * 0.0.0.0/0 10.42.0.17 / rule to permit the traffic traffic to pods when source is the pod’s local node / ADDRTYPE match src-type LOCAL
0 0 KUBE-NWPLCY-DEFAULT all – * * 10.42.0.17 0.0.0.0/0 / run through default egress network policy chain /
0 0 KUBE-NWPLCY-DEFAULT all – * * 0.0.0.0/0 10.42.0.17 / run through default ingress network policy chain /
0 0 NFLOG all – * * 0.0.0.0/0 0.0.0.0/0 / rule to log dropped traffic POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system / mark match ! 0x10000/0x10000 limit: avg 10/min burst 10 nflog-group 100
0 0 REJECT all – * * 0.0.0.0/0 0.0.0.0/0 / rule to REJECT traffic destined for POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system / mark match ! 0x10000/0x10000 reject-with icmp-port-unreachable
0 0 MARK all – * * 0.0.0.0/0 0.0.0.0/0 MARK and 0xfffeffff
0 0 MARK all – * * 0.0.0.0/0 0.0.0.0/0 / set mark to ACCEPT traffic that comply to network policies */ MARK or 0x20000

Chain KUBE-POD-FW-OFNIXDUYRYV7IKTC (7 references)
pkts bytes target prot opt in out source destination
2718 459K ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 /* rule for stateful firewall for pod / ctstate RELATED,ESTABLISHED
2 80 DROP all – * * 0.0.0.0/0 0.0.0.0/0 / rule to drop invalid state for pod / ctstate INVALID
58 3480 ACCEPT all – * * 0.0.0.0/0 10.42.0.15 / rule to permit the traffic traffic to pods when source is the pod’s local node / ADDRTYPE match src-type LOCAL
1 60 KUBE-NWPLCY-DEFAULT all – * * 10.42.0.15 0.0.0.0/0 / run through default egress network policy chain /
0 0 KUBE-NWPLCY-DEFAULT all – * * 0.0.0.0/0 10.42.0.15 / run through default ingress network policy chain /
0 0 NFLOG all – * * 0.0.0.0/0 0.0.0.0/0 / rule to log dropped traffic POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system / mark match ! 0x10000/0x10000 limit: avg 10/min burst 10 nflog-group 100
0 0 REJECT all – * * 0.0.0.0/0 0.0.0.0/0 / rule to REJECT traffic destined for POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system / mark match ! 0x10000/0x10000 reject-with icmp-port-unreachable
1 60 MARK all – * * 0.0.0.0/0 0.0.0.0/0 MARK and 0xfffeffff
1 60 MARK all – * * 0.0.0.0/0 0.0.0.0/0 / set mark to ACCEPT traffic that comply to network policies */ MARK or 0x20000

Chain KUBE-POD-FW-RQOP5FO3X2U2YBDR (7 references)
pkts bytes target prot opt in out source destination
342 42625 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 /* rule for stateful firewall for pod / ctstate RELATED,ESTABLISHED
0 0 DROP all – * * 0.0.0.0/0 0.0.0.0/0 / rule to drop invalid state for pod / ctstate INVALID
21 1260 ACCEPT all – * * 0.0.0.0/0 10.42.0.16 / rule to permit the traffic traffic to pods when source is the pod’s local node / ADDRTYPE match src-type LOCAL
0 0 KUBE-NWPLCY-DEFAULT all – * * 10.42.0.16 0.0.0.0/0 / run through default egress network policy chain /
0 0 KUBE-NWPLCY-DEFAULT all – * * 0.0.0.0/0 10.42.0.16 / run through default ingress network policy chain /
0 0 NFLOG all – * * 0.0.0.0/0 0.0.0.0/0 / rule to log dropped traffic POD name:traefik-69d6585798-f2tm8 namespace: kube-system / mark match ! 0x10000/0x10000 limit: avg 10/min burst 10 nflog-group 100
0 0 REJECT all – * * 0.0.0.0/0 0.0.0.0/0 / rule to REJECT traffic destined for POD name:traefik-69d6585798-f2tm8 namespace: kube-system / mark match ! 0x10000/0x10000 reject-with icmp-port-unreachable
0 0 MARK all – * * 0.0.0.0/0 0.0.0.0/0 MARK and 0xfffeffff
0 0 MARK all – * * 0.0.0.0/0 0.0.0.0/0 / set mark to ACCEPT traffic that comply to network policies */ MARK or 0x20000

Chain KUBE-PROXY-CANARY (0 references)
pkts bytes target prot opt in out source destination

Chain KUBE-PROXY-FIREWALL (3 references)
pkts bytes target prot opt in out source destination

Chain KUBE-ROUTER-FORWARD (1 references)
pkts bytes target prot opt in out source destination
74 21379 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 0.0.0.0/0 10.42.0.16 /* rule to jump traffic destined to POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 0.0.0.0/0 10.42.0.16 PHYSDEV match --physdev-is-bridged / rule to jump traffic destined to POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
77 6043 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 10.42.0.16 0.0.0.0/0 / rule to jump traffic from POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 10.42.0.16 0.0.0.0/0 PHYSDEV match --physdev-is-bridged / rule to jump traffic from POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 0.0.0.0/0 10.42.0.17 / rule to jump traffic destined to POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 0.0.0.0/0 10.42.0.17 PHYSDEV match --physdev-is-bridged / rule to jump traffic destined to POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 10.42.0.17 0.0.0.0/0 / rule to jump traffic from POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 10.42.0.17 0.0.0.0/0 PHYSDEV match --physdev-is-bridged / rule to jump traffic from POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
106 48939 KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 0.0.0.0/0 10.42.0.15 / rule to jump traffic destined to POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC /
0 0 KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 0.0.0.0/0 10.42.0.15 PHYSDEV match --physdev-is-bridged / rule to jump traffic destined to POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC /
100 13883 KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 10.42.0.15 0.0.0.0/0 / rule to jump traffic from POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC /
0 0 KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 10.42.0.15 0.0.0.0/0 PHYSDEV match --physdev-is-bridged / rule to jump traffic from POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC */

Chain KUBE-ROUTER-INPUT (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all – * * 0.0.0.0/0 10.43.0.0/16 /* allow traffic to primary/secondary cluster IP range - EKROEGTNIJ3AP3LC /
0 0 RETURN tcp – * * 0.0.0.0/0 0.0.0.0/0 / allow LOCAL TCP traffic to node ports - LR7XO7NXDBGQJD2M / ADDRTYPE match dst-type LOCAL multiport dports 30000:32767
0 0 RETURN udp – * * 0.0.0.0/0 0.0.0.0/0 / allow LOCAL UDP traffic to node ports - 76UCBPIZNGJNWNUZ / ADDRTYPE match dst-type LOCAL multiport dports 30000:32767
105 8505 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 10.42.0.16 0.0.0.0/0 / rule to jump traffic from POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 10.42.0.17 0.0.0.0/0 / rule to jump traffic from POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
1262 284K KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 10.42.0.15 0.0.0.0/0 / rule to jump traffic from POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC */

Chain KUBE-ROUTER-OUTPUT (1 references)
pkts bytes target prot opt in out source destination
107 7958 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 0.0.0.0/0 10.42.0.16 /* rule to jump traffic destined to POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-RQOP5FO3X2U2YBDR all – * * 10.42.0.16 0.0.0.0/0 / rule to jump traffic from POD name:traefik-69d6585798-f2tm8 namespace: kube-system to chain KUBE-POD-FW-RQOP5FO3X2U2YBDR /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 0.0.0.0/0 10.42.0.17 / rule to jump traffic destined to POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
0 0 KUBE-POD-FW-4KKVF6GINJH2NPO6 all – * * 10.42.0.17 0.0.0.0/0 / rule to jump traffic from POD name:svclb-traefik-8a7362f7-6m956 namespace: kube-system to chain KUBE-POD-FW-4KKVF6GINJH2NPO6 /
1311 116K KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 0.0.0.0/0 10.42.0.15 / rule to jump traffic destined to POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC /
0 0 KUBE-POD-FW-OFNIXDUYRYV7IKTC all – * * 10.42.0.15 0.0.0.0/0 / rule to jump traffic from POD name:metrics-server-7b58468669-hkvk5 namespace: kube-system to chain KUBE-POD-FW-OFNIXDUYRYV7IKTC */

Chain KUBE-SERVICES (2 references)
pkts bytes target prot opt in out source destination

Node2

[root@k8s0007 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether fe:fc:fe:7c:2b:3b brd ff:ff:ff:ff:ff:ff
inet 10.89.64.73/24 brd 10.89.64.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::fcfc:feff:fe7c:2b3b/64 scope link
valid_lft forever preferred_lft forever
3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether 36:00:73:cf:8b:1e brd ff:ff:ff:ff:ff:ff
inet 10.42.1.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::3400:73ff:fecf:8b1e/64 scope link
valid_lft forever preferred_lft forever
4: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 52:07:f8:70:bb:d7 brd ff:ff:ff:ff:ff:ff
inet 10.42.1.1/24 brd 10.42.1.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::5007:f8ff:fe70:bbd7/64 scope link
valid_lft forever preferred_lft forever
5: vethe51e62b1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 26:b0:78:aa:8d:76 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::24b0:78ff:feaa:8d76/64 scope link
valid_lft forever preferred_lft forever
[root@k8s0007 ~]# ip route
default via 10.89.64.1 dev eth0 metric 103
10.42.0.0/24 via 10.42.0.0 dev flannel.1 onlink
10.42.1.0/24 dev cni0 proto kernel scope link src 10.42.1.1
10.42.2.0/24 via 10.42.2.0 dev flannel.1 onlink
10.42.3.0/24 via 10.42.3.0 dev flannel.1 onlink
10.42.4.0/24 via 10.42.4.0 dev flannel.1 onlink
10.89.64.0/24 dev eth0 proto kernel scope link src 10.89.64.73
169.254.0.0/16 dev eth0 scope link metric 1002

[/details]