查看host/container veth pair 关系

1 问题背景

容器中的eth0实际上和外面host上的某个veth是pair关系，然后通过bridge，如docker0 实现在一个host上容器间通信。但是有没有一个办法可以知道host上的vethxxx,到底是和哪个container eth0 是pair关系呢？

2 实现思路

2.1 思路1

容器里面，查看

# cat /sys/class/net/eth0/iflink
5

host上遍历/sys/claas/net下面的全部目录查看子目录ifindex的值和容器里面查出来的iflink值相当的veth名字，也就是：veth63a89a3。这样就找到了container 和 veth pair的关系。

# cat /sys/class/net/veth63a89a3/ifindex
5

2.1 思路2

容器里执行，ip link show eth0 命令，然后可以看到 116: eth0@if117，其中116是eth0接口的index, 117是和他pair的veth的index

~ $ ip link show eth0
116: eth0@if117: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:0a:00:04:08 brd ff:ff:ff:ff:ff:ff

在host执行下面命令可以看到对应117的vethinterface是哪一个，这样就得到了container和veth pair关系

# ip link show | grep 117
117: veth145042b@if116: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue master docker0 state UP

3. 代码实现

不管那种思路，总有一个问题，要能进入到容器中，进入容器有一个比较方便的方法是用nscenter 。因为nscenter安装比较烦，需要下载源代码然后编译等，这里直接将编译好的nscenter 编译出docker image，运行container的时候，会把相关的文件copy 到/user/bin/local 下面。nscenter 参考了： https://github.com/jpetazzo/nsenter

dockerveth 具体代码如下，代码思路蛮简单的，主要是要区别不同的网络模式。

# /bin/bash
get_network_mode() {
    docker inspect --format='{{.HostConfig.NetworkMode}}' "$1"
}

created_by_kubelet() {
    [[ $(docker inspect --format='{{.Name}}' "$1") =~ ^/k8s_ ]]
}

for container_name in $(docker ps --format '{{.Names}}'); do
    container_id=$(docker ps -aqf "name=${container_name}")
	network_mode=$(get_network_mode "${container_id}")
    # skip the containers whose network_mode is 'host' or 'none',
    # but do NOT skip the container created by kubelet.
    if [[ "${network_mode}" == "host" || \
          $(! created_by_kubelet "${container_id}") && "${network_mode}" == "none" ]]; then
        echo "${container_id} => ${network_mode}"
        continue
    fi

    # if one container's network_mode is 'other container',
    # then get its root parent container's network_mode.
    while grep container <<< "${network_mode}" -q; do
        network_mode=$(get_network_mode "${network_mode/container:/}")
        # skip the containers whose network_mode is 'host' or 'none',
        # but do NOT skip the container created by kubelet.
        if [[ "${network_mode}" == "host" || \
              $(! created_by_kubelet "${container_id}") && "${network_mode}" == "none" ]]; then
            echo "${container_id} => ${network_mode}"
            continue 2
        fi
    done

    # get current container's 'container_id'.
    pid=$(docker inspect --format='{{.State.Pid}}' "${container_id}")

    # get the 'id' of veth device in the container.
    veth_id=$(nsenter -t "${pid}" -n ip link show eth0 |grep -oP '(?<=eth0@if)\d+(?=:)')

    # get the 'name' of veth device in the 'docker0' bridge (or other name),
    # which is the peer of veth device in the container.
    veth_name=$(ip link show |sed -nr "s/^${veth_id}: *([^ ]*)@if.*/\1/p")

    echo "${container_id} => ${veth_name} => ${container_name}"
done

编译Dockefile

FROM jpetazzo/nsenter
ADD installer /installer
ADD dockerveth /dockerveth

installer 是：

#!/bin/sh
if mountpoint -q /target; then
	echo "Installing nsenter to /target"
	cp /nsenter /target
	echo "Installing docker-enter to /target"
	cp /docker-enter /target
	echo "Installing importenv to /target"
	cp /importenv /target
	echo "Installing dockerveth to /target"
	cp /dockerveth /target
else
	echo "/target is not a mountpoint."
	echo "You can either:"
	echo "- re-run this container with -v /usr/local/bin:/target"
	echo "- extract the nsenter binary (located at /nsenter)"
fi

4. 如何用

only run once below command by root:

# docker run --rm -v /usr/local/bin:/target shufanhao/dockerveth:v1.0

然后需要的时候运行下dockerveth：

root@szwg-ecom-jpaas08:~/veth# dockerveth
CONTAINER ID	VETH       	NAMES
3b16162ea818	veth145042b	dib_agent_jpaas
f501a753a9da	veth0a4537c	dib_service_jpaas
de653c67370c	vethd28f2ef	dib_jpaas_redis
cde9e15f0f80	veth13e141e	dib_agent_test
0eba4d6fc7c2	vethdd2c3d7	dib_service_test

5 Issue

permission问题如下：
nsenter: cannot open /proc/2609/ns/net: Permission denied
解决： sudo dockerveth // 用root账户运行
脚本执行错误如下：
Syntax error: redirection unexpected
解决：bash dockerveth