使用 Linux bridge/netns 连接 Docker 容器

本文创建 docker 容器指定参数 --net=none，基于 Linux Bridge 和 netns 技术为容器配置 IP 地址，实现容器和外界通信。

架构

示例架构：通过 Bridge 连接两个 Docker 容器 的 tap 虚拟网卡和物理网卡 eth0

搭建步骤

创建网桥

• 创建网桥 br0，并为 br0 设置IP为 172.20.0.1/16，网段为 172.20.0.0/16

$ ip link add br0 type bridge
$ ip link show br0  # 查看网络设备（network device）信息
4: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether ca:bf:0e:08:d7:65 brd ff:ff:ff:ff:ff:ff
$ ip link set br0 up
$ ip link show br0
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ca:bf:0e:08:d7:65 brd ff:ff:ff:ff:ff:ff
$ ip addr add 172.20.0.1/16 dev br0
$ ip addr show br0
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 06:d9:35:e0:9e:e1 brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.1/16 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::c8bf:eff:fe08:d765/64 scope link
       valid_lft forever preferred_lft forever

此时可以看到宿主机路由多了一条到 br0 的信息

$ ip route
172.20.0.0/16 dev br0 proto kernel scope link src 172.20.0.1

也可以在宿主机 ping 通 172.20.0.1。

创建容器

• 创建 --net=none 容器

$ docker run -itd --privileged --net=none --name c7-1 xiexianbin/centos:7-net /sbin/init
cad156ef8d4dff00abc5927c8ff0d9c507147016272f08738852cb321ee6052d

创建 netns

$ pid=$(docker inspect -f '{{.State.Pid}}' c7-1)  # 获取容器的pid
$ file /proc/3489/ns/net  # 容器的网
/proc/3489/ns/net: broken symbolic link to `net:[4026532452]'
$ sudo mkdir -p /var/run/netns
$ sudo ln -s /proc/$pid/ns/net /var/run/netns/$pid
# ip netns  # 查看network namespace信息
3489

创建和配置 veth pair

• 创建 veth pair，c7a 和 c7b

$ sudo ip link add c7a type veth peer name c7b
$ ip link
...
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ca:bf:0e:08:d7:65 brd ff:ff:ff:ff:ff:ff
13: c7b@c7a: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether ba:2f:78:24:ad:d1 brd ff:ff:ff:ff:ff:ff
14: c7a@c7b: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 06:d9:35:e0:9e:e1 brd ff:ff:ff:ff:ff:ff

• 将 c7a 添加到网桥 br0

$ sudo brctl addif br0 c7a
$ sudo ip link set c7a up
$ brctl show
bridge name	bridge id		STP enabled	interfaces
br0		8000.06d935e09ee1	no		c7a

• 将 c7b 绑定到 netns，命名为 eth0，启动它

$ sudo ip link set c7b netns $pid
$ ip netns exec $pid ip a
...
13: c7b@if14: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ba:2f:78:24:ad:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
$ sudo ip netns exec $pid ip link set dev c7b name eth0  # 改名 c7b 为 eth0
# sudo ip netns exec $pid ip link set dev eth0 up  # 启动 eth0
$ ip netns exec $pid ip a
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:2f:78:24:ad:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0

• 为 eth0 配置一个可用 IP（172.20.0.2/16网段）和默认网关（172.20.0.1）

$ sudo ip netns exec $pid ip addr add 172.20.0.2/16 dev eth0
$ sudo ip netns exec $pid ip route add default via 172.20.0.1
$ sudo ip netns exec $pid ip add show
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:2f:78:24:ad:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.20.0.2/16 scope global eth0
       valid_lft forever preferred_lft forever

验证

• 重启内确认，可以正常ping通网关

$ docker exec -it c7-1 bash
[root@cad156ef8d4d /]# ip a
...
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:2f:78:24:ad:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.20.0.2/16 scope global eth0
       valid_lft forever preferred_lft forever
[root@cad156ef8d4d /]# ping -c 1 172.20.0.1
PING 172.20.0.1 (172.20.0.1) 56(84) bytes of data.
64 bytes from 172.20.0.1: icmp_seq=1 ttl=64 time=0.126 ms

--- 172.20.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.126/0.126/0.126/0.000 ms