Use Linux bridge to manually build docker container network

1. Run the container, add static routing in busybox requires real root privileges, so add--privileged=true

$ docker run --rm --name box1 -d --net none --privileged=true busybox sleep 9999

$ docker run --rm --name box2 -d --net none --privileged=true busybox sleep 9999

2. View process PID

$ docker inspect box1|grep Pid
            "Pid": 2002,
            "PidMode": "",
            "PidsLimit": null,

$ docker inspect box2|grep Pid
            "Pid": 2063,
            "PidMode": "",
            "PidsLimit": null,

3. Expose the network namespace of the container to the system

$ mkdir -p /var/run/netns

$ ln -s /proc/2002/ns/net /var/run/netns/box1

$ ln -s /proc/2063/ns/net /var/run/netns/box2

4. View the network namespace of the container

$ ip netns exec box1 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
       
$ ip netns exec box2 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo

5. Create bridges and veth-pairs, and configure IP addresses for them

$ brctl addbr br0
$ ip addr add 10.0.0.1/24 dev br0
$ ip link add v1 type veth peer name v1_
$ ip link add v2 type veth peer name v2_
$ ip link set v1_ netns box1
$ ip link set v2_ netns box2
$ brctl addif br0 v1
$ brctl addif br0 v2
$ ip netns exec box1 ip addr add 10.0.0.2/24 dev v1_
$ ip netns exec box2 ip addr add 10.0.0.3/24 dev v2_
$ 
$ ip link set br0 up
$ ip link set v1 up
$ ip link set v2 up
$ ip netns exec box1 ip link set v1_ up
$ ip netns exec box2 ip link set v2_ up

6. Enter the container to configure the static route, check the network status, 192.168.0.122 is the host network card IP. At this point, the two containers can ping each other, and both the gateway and the host's network card IP can be pinged.

$ docker exec -it box1 sh
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
5: v1_@if6: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue qlen 1000
    link/ether 22:db:98:1a:f1:cb brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 scope global v1_
       valid_lft forever preferred_lft forever

/ # route add default gw 10.0.0.1

/ # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 v1_
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 v1_

/ # ping -c 1 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.070 ms

--- 10.0.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.070/0.070/0.070 ms
/ # ping -c 1 10.0.0.2
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: seq=0 ttl=64 time=0.126 ms

--- 10.0.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.126/0.126/0.126 ms
/ # ping -c 1 10.0.0.3
PING 10.0.0.3 (10.0.0.3): 56 data bytes
64 bytes from 10.0.0.3: seq=0 ttl=64 time=0.167 ms

--- 10.0.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.167/0.167/0.167 ms

/ # ping -c 1 192.168.0.122
PING 192.168.0.122 (192.168.0.122): 56 data bytes
64 bytes from 192.168.0.122: seq=0 ttl=64 time=0.070 ms

--- 192.168.0.122 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.070/0.070/0.070 ms

7. Configure nat on the host

$ iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o ens33 -j MASQUERADE

8. In the container, you can ping the host's gateway and external network

/ # ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=63 time=1.143 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.143/1.143/1.143 ms
/ # ping -c 1 www.baidu.com
PING www.baidu.com (61.135.169.121): 56 data bytes
64 bytes from 61.135.169.121: seq=0 ttl=56 time=6.209 ms

--- www.baidu.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 6.209/6.209/6.209 ms