Article directory
foreword
在云原生领域，Kubernetes 已经成为了最主流的容器管理工具。Kubernetes 支持将容器部署到多个节点（即主机）上，因此必须解决容器间跨主机通信的问题。
This article will introduce in detail the principle and method of container cross-host network in Kubernetes.
What is Flannel
-
In Kubernetes, Flannel is an open source container networking solution. It is responsible for creating a virtual network in the Kubernetes cluster so that Pods on different nodes can use the same IP address to communicate.
-
Flannel aims to provide simple, fast, and reliable container networking.
-
Flannel achieves network isolation by assigning each node a unique subnet, and then having Pods between nodes communicate over these subnets. Flannel supports a variety of backends, including UDP, VXLAN, and Host-gw.
What are the backend implementations of Flannel?
The backend implementation of Flannel mainly includes the following methods:
- VXLAN；
- host-gw；
- UDP。
These three different backend implementations represent the mainstream implementation methods for three containers across the main network.
Among these three modes, the UDP mode can be said to be a mode supported by the Flannel project earlier, but it is also the mode with the worst performance.
Therefore, this mode has been used less so far. However, there is a reason why Flannel chose the UDP mode first, because this mode is the most direct and easy to understand container cross-main network implementation.
UDP
UDP is the default backend implementation of Flannel. It uses the UDP protocol to create a set of overlay networks to connect all nodes in the Kubernetes cluster. Each node gets a subnet and assigns IP addresses from it to the pods running on it.
When a Pod needs to communicate with a Pod on another node, it encapsulates the data packet in a UDP packet, sends it over the overlay network to the target node, and unpacks it there. Due to UDP's light weight and simplicity, it is the most commonly used backend implementation for Flannel.
VXLAN
VXLAN is a virtualized tunneling protocol that passes Ethernet traffic between different subnets. Flannel uses VXLAN to create an overlay network, and each node will obtain a unique VTEP (VXLAN Tunnel Endpoint), and assign its IP address to the Pod running on it.
Compared with UDP, VXLAN requires more computing resources, but it provides better reliability and flexibility.
Host-gw
Host-gw is a "host gateway" mode that treats each node as a gateway, routing all pod traffic directly to the subnet of the pod on the target node.
This method is very simple because it does not require any overlay network and encapsulation and decapsulation process. However, it needs to configure the routing table on all nodes correctly, and also needs to dynamically change the routing table in the cluster to ensure the communication between containers.
Cross-master communication based on Flannel UDP mode
-
In Flannel's UDP mode, each node gets a unique subnet and assigns IP addresses within it to the pods running on it. When a Pod needs to communicate with a Pod on another node, it encapsulates the data packet in a UDP packet, sends it over the overlay network to the target node, and unpacks it there.
-
Specifically, Flannel initially starts a flanneld service on each node in the cluster. The flanneld service coordinates the overlay network between all nodes and assigns each node a unique subnet.
-
The kubelet service running on each node then assigns an IP address from that subnet to the container when it starts. Applications inside the container can use this IP address to access other containers in the cluster.
UDP mode case implementation
Suppose you have a Kubernetes cluster with two nodes, and each node has a container running. The container names are "web1" and "web2" and they are both running on different nodes. Now, we want to enable cross-host communication between these two containers.
- First, we need to make sure that Flannel has been installed on each node and the flanneld service has been successfully started. Then, run the following command on each node to check that Flannel is working:
$ sudo systemctl status flanneld
- Next, we need to assign each node a unique subnet. Suppose we set the subnet of the first node to 10.244.0.0/24 and the subnet of the second node to 10.244.1.0/24. Run the following commands on each node to configure Flannel and assign subnets:
$ sudo vim /etc/sysconfig/flanneld
FLANNEL_ETCD="http://etcd-ip:2379"
FLANNEL_ETCD_PREFIX="/kube-centos/network"
FLANNEL_OPTIONS="-iface=eth0"
$ sudo systemctl restart flanneld
where etcd-ip is the IP address of the etcd server. Note that the value of FLANNEL_ETCD_PREFIX must be the same for all nodes so that they get the same subnet information from etcd.
- We can now communicate within the container using the specified IP address. For example, run the following command inside the web1 container to send a ping packet to the web2 container:
$ ping 10.244.1.2
where 10.244.1.2 is the IP address of the web2 container.
Through the Flannel UDP mode, we can easily achieve cross-host communication, making Kubernetes-based applications run more stably and reliably.
The basic principle of cross-master communication based on Flannel UDP mode is as follows:
Cross-master communication based on Flannel VXLAN mode
In Flannel's VXLAN mode, each node will obtain a unique VTEP, which can encapsulate container data packets in VXLAN packets and transmit them in the overlay network. Unlike UDP mode, VXLAN mode requires the use of VXLAN protocol for encapsulation and decapsulation.
Specifically, Flannel VNI (VXLAN Network Identifier) will be assigned to the overlay network, and each node must configure a VTEP (VXLAN Tunnel Endpoint) for the network.
Additionally, each Pod will be given a unique IP address within that network, which it will use to communicate with other Pods.
VXLAN mode case implementation
Suppose you have a Kubernetes cluster with two nodes, and each node has a container running. The container names are "web1" and "web2" and they are both running on different nodes. Now, we want to enable cross-host communication between these two containers.
- First, we need to make sure that Flannel has been installed on each node and the flanneld service has been successfully started. Then, run the following command on each node to check that Flannel is working:
$ sudo systemctl status flanneld
- Next, we need to assign each node a unique VTEP. Assume we set the VXLAN Network ID to 1, the VTEP IP address of the first node is 192.168.0.1, and the VTEP IP address of the second node is 192.168.0.2. Run the following commands on each node to configure Flannel and assign VTEPs:
$ sudo vim /etc/sysconfig/flanneld
FLANNEL_ETCD="http://etcd-ip:2379"
FLANNEL_ETCD_PREFIX="/kube-centos/network"
FLANNEL_OPTIONS="-iface=eth0 -vni=1"
$ sudo ifconfig flannel.1 192.168.0.1/24 up
$ sudo systemctl restart flanneld
- Cross-host communication is then possible within each container. For example, run the following command inside the web1 container to send a ping packet to the web2 container:
$ ping 10.244.1.2
where 10.244.1.2 is the IP address of the web2 container.
Through the Flannel VXLAN mode, we can easily achieve cross-host communication and provide better reliability and flexibility. When running Kubernetes-based large-scale applications, using Flannel VXLAN mode can effectively improve network performance and communication efficiency.
The basic principle of cross-master communication based on Flannel VXLAN mode is as follows:
Summarize
The above mainly introduces在 Kubernetes 中容器跨主机网络的实现原理和方法。
-
As a container network solution supported by Kubernetes, Flannel has become one of the most popular container network solutions in the cloud native field.
-
Flannel transmits container packets in the overlay network by assigning each node a unique subnet or VTEP, so that Pods on different nodes can communicate using the same IP address.
-
Flannel provides a variety of backend implementations, including UDP, VXLAN, and Host-gw. Regardless of the backend implementation, Flannel can help us easily achieve cross-host communication and provide a stable and efficient container network solution.