Home Archives Categories Tags

OpenStack 安全组(Security Group) 默认是通过 Linux iptables 实现的,本文介绍 Neutron 基于 iptables 实现的安全组。

启用 iptables 安全组

修改 /etc/neutron/plugins/ml2/ml2_conf.ini 配置文件

[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

重启服务

sudo systemctl restart devstack@q-*

默认的 iptables 规则

通过 iptables-save 命令可以看到的流被转发到 neutron-openvswi-x 的链中:

$ sudo iptables-save
...
-A INPUT -j neutron-openvswi-INPUT  # 将 INPUT 转到 neutron-openvswi-INPUT 链
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD  # 将 FORWARD 转到 neutron-openvswi-FORWARD 链
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-openvswi-OUTPUT  # 将 OUTPUT 转到 neutron-openvswi-OUTPUT 链
-A neutron-filter-top -j neutron-openvswi-local
...

查看 Chain 信息如下:

default-iptables-rule
stack@xiexianbin-vm:~$ sudo iptables -nvL
Chain INPUT (policy ACCEPT 24762 packets, 6088K bytes)
 pkts bytes target     prot opt in     out     source               destination
24762 6088K neutron-openvswi-INPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0
58550   14M LIBVIRT_INP  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 neutron-filter-top  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 neutron-openvswi-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 LIBVIRT_FWX  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 LIBVIRT_FWI  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 LIBVIRT_FWO  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 24381 packets, 5993K bytes)
 pkts bytes target     prot opt in     out     source               destination
24381 5993K neutron-filter-top  all  --  *      *       0.0.0.0/0            0.0.0.0/0
24381 5993K neutron-openvswi-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0
57654   14M LIBVIRT_OUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain LIBVIRT_FWI (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
    0     0 REJECT     all  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain LIBVIRT_FWO (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0
    0     0 REJECT     all  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain LIBVIRT_FWX (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0

Chain LIBVIRT_INP (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:67

Chain LIBVIRT_OUT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    0     0 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:68
    0     0 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            tcp dpt:68

Chain neutron-filter-top (2 references)
 pkts bytes target     prot opt in     out     source               destination
24381 5993K neutron-openvswi-local  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain neutron-openvswi-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain neutron-openvswi-INPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain neutron-openvswi-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain neutron-openvswi-local (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain neutron-openvswi-sg-chain (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain neutron-openvswi-sg-fallback (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Default drop rule for unmatched traffic. */

默认初始化的 iptables Chain 链如下:

  • neutron-filter-top
  • neutron-openvswi-FORWARD # neutron 定义的 FORWARD 链
  • neutron-openvswi-INPUT # neutron 定义的 INPUT 链
  • neutron-openvswi-OUTPUT # neutron 定义的 OUTPUT 链
  • neutron-openvswi-local
  • neutron-openvswi-sg-chain
  • neutron-openvswi-sg-fallback

创建网络和子网

  • 创建网络
create-network
stack@xiexianbin-vm:/etc/neutron$ openstack network create demo --share --enable
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2021-01-03T00:59:49Z                 |
| description               |                                      |
| dns_domain                | None                                 |
| id                        | 557bc377-cece-41dd-8b77-1348355ff66c |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | False                                |
| is_vlan_transparent       | None                                 |
| mtu                       | 1450                                 |
| name                      | demo                                 |
| port_security_enabled     | True                                 |
| project_id                | 45837e6267c44d7788a919d6e342e64c     |
| provider:network_type     | vxlan                                |
| provider:physical_network | None                                 |
| provider:segmentation_id  | 1                                    |
| qos_policy_id             | None                                 |
| revision_number           | 1                                    |
| router:external           | Internal                             |
| segments                  | None                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2021-01-03T00:59:49Z                 |
+---------------------------+--------------------------------------+
stack@xiexianbin-vm:~$ openstack network list
+--------------------------------------+------+---------+
| ID                                   | Name | Subnets |
+--------------------------------------+------+---------+
| 557bc377-cece-41dd-8b77-1348355ff66c | demo |         |
+--------------------------------------+------+---------+
  • 创建子网,并启用 --dhcp
create-subnet
stack@xiexianbin-vm:/etc/neutron$ openstack subnet create sub-demo --subnet-range 10.0.0.0/24 --dhcp --ip-version 4 --network demo
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.0.0.2-10.0.0.254                  |
| cidr                 | 10.0.0.0/24                          |
| created_at           | 2021-01-03T01:00:48Z                 |
| description          |                                      |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.0.0.1                             |
| host_routes          |                                      |
| id                   | 3768efcb-e21e-4293-a1f9-596e28a21cb5 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | sub-demo                             |
| network_id           | 557bc377-cece-41dd-8b77-1348355ff66c |
| prefix_length        | None                                 |
| project_id           | 45837e6267c44d7788a919d6e342e64c     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2021-01-03T01:00:48Z                 |
+----------------------+--------------------------------------+
stack@xiexianbin-vm:/etc/neutron$ openstack subnet list
stack@xiexianbin-vm:~$ openstack subnet list
+--------------------------------------+----------+--------------------------------------+-------------+
| ID                                   | Name     | Network                              | Subnet      |
+--------------------------------------+----------+--------------------------------------+-------------+
| 3768efcb-e21e-4293-a1f9-596e28a21cb5 | sub-demo | 557bc377-cece-41dd-8b77-1348355ff66c | 10.0.0.0/24 |
+--------------------------------------+----------+--------------------------------------+-------------+
stack@xiexianbin-vm:~$ openstack port list
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                      | Status |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+
| 190045e5-4415-4599-bea2-9eb5a8f73b7d |      | fa:16:3e:a3:28:f3 | ip_address='10.0.0.2', subnet_id='3768efcb-e21e-4293-a1f9-596e28a21cb5' | ACTIVE |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+

可以发现已经分配了一个ip地址 10.0.0.2,该地址为 dhcp 的地址,接下来我们从 netnsOVS 等方面查看他的信息。

netns 信息

创建子网(并启用dhcp)时,创建的 netns 信息如下,可以看到 tap190045e5-44 已经在该 netns 中,对应的地址为: 10.0.0.2/24

default-ip-netns
stack@xiexianbin-vm:~$ ip netns
qdhcp-557bc377-cece-41dd-8b77-1348355ff66c (id: 0)
stack@xiexianbin-vm:~$ sudo ip netns exec qdhcp-557bc377-cece-41dd-8b77-1348355ff66c ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
10: tap190045e5-44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:16:3e:a3:28:f3 brd ff:ff:ff:ff:ff:ff
stack@xiexianbin-vm:~$ sudo ip netns exec qdhcp-557bc377-cece-41dd-8b77-1348355ff66c ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
10: tap190045e5-44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:a3:28:f3 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global tap190045e5-44
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fea3:28f3/64 scope link
       valid_lft forever preferred_lft forever

OVS 信息

tap190045e5-44 绑定在 br-int 网桥上,tag1,为该网络提供 dhcp 功能。

default-ovs-vsctl-show
stack@xiexianbin-vm:~$ sudo ovs-vsctl show
0976470f-8041-45d1-ad1a-34cd061684d3
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-ex
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-int
            Interface br-int
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port tap190045e5-44
            tag: 1
            Interface tap190045e5-44
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    ovs_version: "2.13.1"
stack@xiexianbin-vm:~$ sudo ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000a62b2fabc24d
n_tables:254, n_buffers:0
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
 1(int-br-ex): addr:3e:e4:b9:12:01:57
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:96:5e:3a:be:41:08
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(tap190045e5-44): addr:fa:16:3e:a3:28:f3
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:a6:2b:2f:ab:c2:4d
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

iptables 信息

通过 iptables 规则,默认把所有流量转发到 tap190045e5-44iptables 信息如下:

stack@xiexianbin-vm:~$ sudo iptables -nvL
Chain neutron-openvswi-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap190045e5-44 --physdev-is-bridged /* Accept all packets when port is trusted. */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tap190045e5-44 --physdev-is-bridged /* Accept all packets when port is trusted. */

neutron-dhcp-agent 通过该 tap 设备为该 network 提供 dhcp 功能。TODO(xiexianbin)

安全组信息

默认情况下,对所有 ipv4/6 出的流量和本安全组内的流量不限制:

default-security-group
stack@xiexianbin-vm:~$ openstack security group list
+--------------------------------------+---------+------------------------+----------------------------------+------+
| ID                                   | Name    | Description            | Project                          | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+
| 22b33e63-0f84-4bbc-a797-7c3d7ae107aa | default | Default security group | 45837e6267c44d7788a919d6e342e64c | []   |
+--------------------------------------+---------+------------------------+----------------------------------+------+
stack@xiexianbin-vm:~$ openstack security group rule list 22b33e63-0f84-4bbc-a797-7c3d7ae107aa
+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+
| ID                                   | IP Protocol | Ethertype | IP Range  | Port Range | Remote Security Group                |
+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+
| 09c623fc-bf1f-46bf-9149-facdc6148831 | None        | IPv6      | ::/0      |            | None                                 |
| 0f709d8c-7400-40dd-a5b7-59beaee93c74 | None        | IPv6      | ::/0      |            | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa |
| 366c1188-988b-4c5a-8bf4-dfcec6fb902f | None        | IPv4      | 0.0.0.0/0 |            | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa |
| 6ff01e6d-9b34-433c-968f-c04495098145 | None        | IPv4      | 0.0.0.0/0 |            | None                                 |
+--------------------------------------+-------------+-----------+-----------+------------+--------------------------------------+
stack@xiexianbin-vm:~$ openstack security group show 22b33e63-0f84-4bbc-a797-7c3d7ae107aa
...
| rules           | created_at='2021-01-03T01:25:18Z', direction='egress', ethertype='IPv6', id='09c623fc-bf1f-46bf-9149-facdc6148831', updated_at='2021-01-03T01:25:18Z'                                                          |
|                 | created_at='2021-01-03T01:25:18Z', direction='ingress', ethertype='IPv6', id='0f709d8c-7400-40dd-a5b7-59beaee93c74', remote_group_id='22b33e63-0f84-4bbc-a797-7c3d7ae107aa', updated_at='2021-01-03T01:25:18Z' |
|                 | created_at='2021-01-03T01:25:18Z', direction='ingress', ethertype='IPv4', id='366c1188-988b-4c5a-8bf4-dfcec6fb902f', remote_group_id='22b33e63-0f84-4bbc-a797-7c3d7ae107aa', updated_at='2021-01-03T01:25:18Z' |
|                 | created_at='2021-01-03T01:25:18Z', direction='egress', ethertype='IPv4', id='6ff01e6d-9b34-433c-968f-c04495098145', updated_at='2021-01-03T01:25:18Z'                                                          |
...

创建 VM

creat-cloud-vm
stack@xiexianbin-vm:~$ openstack server create demo-1 --image cirros-0.5.1-x86_64-disk --flavor m1.tiny --security-group 22b33e63-0f84-4bbc-a797-7c3d7ae107aa --network 557bc377-cece-41dd-8b77-1348355ff66c
+-------------------------------------+-----------------------------------------------------------------+
| Field                               | Value                                                           |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                          |
| OS-EXT-AZ:availability_zone         |                                                                 |
| OS-EXT-SRV-ATTR:host                | None                                                            |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                            |
| OS-EXT-SRV-ATTR:instance_name       |                                                                 |
| OS-EXT-STS:power_state              | NOSTATE                                                         |
| OS-EXT-STS:task_state               | scheduling                                                      |
| OS-EXT-STS:vm_state                 | building                                                        |
| OS-SRV-USG:launched_at              | None                                                            |
| OS-SRV-USG:terminated_at            | None                                                            |
| accessIPv4                          |                                                                 |
| accessIPv6                          |                                                                 |
| addresses                           |                                                                 |
| adminPass                           | zA2XHUku6vAM                                                    |
| config_drive                        |                                                                 |
| created                             | 2021-01-03T01:30:53Z                                            |
| flavor                              | m1.tiny (1)                                                     |
| hostId                              |                                                                 |
| id                                  | a9afb91b-8700-4901-9565-5bf6a63f0143                            |
| image                               | cirros-0.5.1-x86_64-disk (000fcfa3-7ffa-4125-817e-cf7a27cf3eb1) |
| key_name                            | None                                                            |
| name                                | demo-1                                                          |
| progress                            | 0                                                               |
| project_id                          | 45837e6267c44d7788a919d6e342e64c                                |
| properties                          |                                                                 |
| security_groups                     | name='22b33e63-0f84-4bbc-a797-7c3d7ae107aa'                     |
| status                              | BUILD                                                           |
| updated                             | 2021-01-03T01:30:52Z                                            |
| user_id                             | c6b926ebfb5c4a6db670150b183581af                                |
| volumes_attached                    |                                                                 |
+-------------------------------------+-----------------------------------------------------------------+
stack@xiexianbin-vm:~$ openstack port list --server a9afb91b-8700-4901-9565-5bf6a63f0143
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                       | Status |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------+--------+
| 33d3e592-c6eb-4cb3-8266-33c8e2e1f646 |      | fa:16:3e:57:1c:cd | ip_address='10.0.0.29', subnet_id='3768efcb-e21e-4293-a1f9-596e28a21cb5' | ACTIVE |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------+--------+
stack@xiexianbin-vm:~$ openstack port show 33d3e592-c6eb-4cb3-8266-33c8e2e1f646
+-------------------------+-------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                       |
+-------------------------+-------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                          |
| allowed_address_pairs   |                                                                                                             |
| binding_host_id         | xiexianbin-vm                                                                                               |
| binding_profile         |                                                                                                             |
| binding_vif_details     | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='True', port_filter='True' |
| binding_vif_type        | ovs                                                                                                         |
| binding_vnic_type       | normal                                                                                                      |
| created_at              | 2021-01-03T01:30:54Z                                                                                        |
| data_plane_status       | None                                                                                                        |
| description             |                                                                                                             |
| device_id               | a9afb91b-8700-4901-9565-5bf6a63f0143                                                                        |
| device_owner            | compute:nova                                                                                                |
| dns_assignment          | None                                                                                                        |
| dns_domain              | None                                                                                                        |
| dns_name                | None                                                                                                        |
| extra_dhcp_opts         |                                                                                                             |
| fixed_ips               | ip_address='10.0.0.29', subnet_id='3768efcb-e21e-4293-a1f9-596e28a21cb5'                                    |
| id                      | 33d3e592-c6eb-4cb3-8266-33c8e2e1f646                                                                        |
| ip_allocation           | None                                                                                                        |
| mac_address             | fa:16:3e:57:1c:cd                                                                                           |
| name                    |                                                                                                             |
| network_id              | 557bc377-cece-41dd-8b77-1348355ff66c                                                                        |
| numa_affinity_policy    | None                                                                                                        |
| port_security_enabled   | True                                                                                                        |
| project_id              | 45837e6267c44d7788a919d6e342e64c                                                                            |
| propagate_uplink_status | None                                                                                                        |
| qos_network_policy_id   | None                                                                                                        |
| qos_policy_id           | None                                                                                                        |
| resource_request        | None                                                                                                        |
| revision_number         | 4                                                                                                           |
| security_group_ids      | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa                                                                        |
| status                  | ACTIVE                                                                                                      |
| tags                    |                                                                                                             |
| trunk_details           | None                                                                                                        |
| updated_at              | 2021-01-03T01:31:00Z                                                                                        |
+-------------------------+-------------------------------------------------------------------------------------------------------------+

发现:

  • 每个 port 均和 security_group_ids 绑定,并且一个 port 上可以绑定多个安全组

查看 OVS 信息

发现 br-int 上新增了一个 qvo33d3e592-c6,该网桥名称为 qvo<port-id-prefix-11> 组成,信息如下:

stack@xiexianbin-vm:~$ sudo ovs-vsctl show
0976470f-8041-45d1-ad1a-34cd061684d3
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-ex
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        datapath_type: system
        Port br-int
            Interface br-int
                type: internal
        Port qvo33d3e592-c6
            tag: 1
            Interface qvo33d3e592-c6
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port tap190045e5-44
            tag: 1
            Interface tap190045e5-44
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    ovs_version: "2.13.1"
stack@xiexianbin-vm:~$ sudo ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000a62b2fabc24d
n_tables:254, n_buffers:0
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
 1(int-br-ex): addr:3e:e4:b9:12:01:57
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:96:5e:3a:be:41:08
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(tap190045e5-44): addr:fa:16:3e:a3:28:f3
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 4(qvo33d3e592-c6): addr:0a:d6:fb:38:47:33
     config:     0
     state:      0
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:a6:2b:2f:ab:c2:4d
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

veth-pair 信息

通过 ip addr show 命令,我们发现 qvo33d3e592-c6veth-pair的一个,另一个为 qvb33d3e592-c6 绑定在 Linux bridge qbr33d3e592-c6

ip-addr-show
stack@xiexianbin-vm:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:4b:60:15 brd ff:ff:ff:ff:ff:ff
    inet 192.168.179.9/24 brd 192.168.179.255 scope global dynamic noprefixroute ens33
       valid_lft 82201sec preferred_lft 82201sec
    inet6 fe80::5d9a:faaa:c5fe:dfb/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: ens38: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:4b:60:1f brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.2/24 brd 172.20.0.255 scope global dynamic noprefixroute ens38
       valid_lft 82201sec preferred_lft 82201sec
    inet6 fe80::2b5:9fdf:5bc3:87b6/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether aa:7c:37:6e:3e:d4 brd ff:ff:ff:ff:ff:ff
5: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b6:97:c7:9b:a9:45 brd ff:ff:ff:ff:ff:ff
6: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
    link/ether a6:2b:2f🆎c2:4d brd ff:ff:ff:ff:ff:ff
7: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 36:77:2d:10:6d:43 brd ff:ff:ff:ff:ff:ff
...
12: qvo33d3e592-c6@qvb33d3e592-c6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 0a:d6:fb:38:47:33 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8d6:fbff:fe38:4733/64 scope link
       valid_lft forever preferred_lft forever
13: qvb33d3e592-c6@qvo33d3e592-c6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master qbr33d3e592-c6 state UP group default qlen 1000
    link/ether 3a:6d:32:cc:34:4a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::386d:32ff:fecc:344a/64 scope link
       valid_lft forever preferred_lft forever

Linux bridge 信息

发现新增了一个 qbr33d3e592-c6,该网桥名称为 qbr<port-id-prefix-11> 组成,基于 iptables 实现的安全组规则将基于此网桥实现。

stack@xiexianbin-vm:~$ sudo ip a show type bridge
11: qbr33d3e592-c6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 3a:6d:32:cc:34:4a brd ff:ff:ff:ff:ff:ff

tap 信息

同样的,通过 ip addr show 命令,我们发现网桥 qbr33d3e592-c6 还绑定这一个 tap 设备TODO(xiexianbin),名称为 tap33d3e592-c6

stack@xiexianbin-vm:~$ ip addr show
...
14: tap33d3e592-c6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel master qbr33d3e592-c6 state UNKNOWN group default qlen 1000
    link/ether fe:16:3e:57:1c:cd brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fe57:1ccd/64 scope link
       valid_lft forever preferred_lft forever
...

云主机使用 tap 设备

通过 ps -ef|grep qemu 查看当前使用qemuTODO(xiexianbin)虚拟化的主机进程为 5369 -netdev tap,fd=32,id=hostnet0,(TODO(xiexianbin)什么是fd)通过如下命令发现,该主机使用的就是 tap33d3e592-c6

stack@xiexianbin-vm:~$ sudo cat /proc/5369/fdinfo/32
pos:	82
flags:	0104002
mnt_id:	27
iff:	tap33d3e592-c6

至此,我们已经将云主机怎么与br-int连接介绍清楚。

iptables 信息

创建 vm 后,iptables 规则变化如下:

new-iptables-rule
$ iptables -nvL
...
Chain neutron-openvswi-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    7  1116 neutron-openvswi-sg-chain  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap33d3e592-c6 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
   13  1504 neutron-openvswi-sg-chain  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tap33d3e592-c6 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap190045e5-44 --physdev-is-bridged /* Accept all packets when port is trusted. */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tap190045e5-44 --physdev-is-bridged /* Accept all packets when port is trusted. */

Chain neutron-openvswi-INPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 neutron-openvswi-o33d3e592-c  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tap33d3e592-c6 --physdev-is-bridged /* Direct incoming traffic from VM to the security group chain. */

Chain neutron-openvswi-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain neutron-openvswi-i33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination
    5   420 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
    2   696 RETURN     udp  --  *      *       0.0.0.0/0            10.0.0.29            udp spt:67 dpt:68
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            255.255.255.255      udp spt:67 dpt:68
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set NIPv422b33e63-0f84-4bbc-a797- src
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
    0     0 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */

Chain neutron-openvswi-o33d3e592-c (2 references)
 pkts bytes target     prot opt in     out     source               destination
    2   658 RETURN     udp  --  *      *       0.0.0.0              255.255.255.255      udp spt:68 dpt:67 /* Allow DHCP client traffic. */
   11   846 neutron-openvswi-s33d3e592-c  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp spt:68 dpt:67 /* Allow DHCP client traffic. */
    0     0 DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68 /* Prevent DHCP Spoofing by VM. */
    4   336 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
    7   510 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
    0     0 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */

Chain neutron-openvswi-s33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination
   11   846 RETURN     all  --  *      *       10.0.0.29            0.0.0.0/0            MAC FA:16:3E:57:1C:CD /* Allow traffic from defined IP/MAC pairs. */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Drop traffic without an IP/MAC allow rule. */

Chain neutron-openvswi-sg-chain (2 references)
 pkts bytes target     prot opt in     out     source               destination
    7  1116 neutron-openvswi-i33d3e592-c  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap33d3e592-c6 --physdev-is-bridged /* Jump to the VM specific chain. */
   13  1504 neutron-openvswi-o33d3e592-c  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in tap33d3e592-c6 --physdev-is-bridged /* Jump to the VM specific chain. */
   20  2620 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0
...

与该主机端口相关的,主要的 iptables Chain 如下:

  • neutron-openvswi-s33d3e592-c:命令 neutron-openvswi-s<port-id-prefix-10>,处理出该端口的网络包的防IP欺骗
  • neutron-openvswi-i33d3e592-c:命令 neutron-openvswi-i<port-id-prefix-10>,处理进该端口的网络包的防IP欺骗
  • neutron-openvswi-o33d3e592-c:命令 neutron-openvswi-o<port-id-prefix-10>,处理出该端口的网络包

以进入端口的链 neutron-openvswi-i33d3e592-c 为例,说明如下:

stack@xiexianbin-vm:~$ sudo iptables -nvL neutron-openvswi-i33d3e592-c
Chain neutron-openvswi-i33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination
    5   420 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */  # 放行已知会话相关联的信息包
    2   696 RETURN     udp  --  *      *       0.0.0.0/0            10.0.0.29            udp spt:67 dpt:68  # 放行到 10.0.0.29 的 utp 包
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            255.255.255.255      udp spt:67 dpt:68  # 放行到 255.255.255.255 的 utp 包
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set NIPv422b33e63-0f84-4bbc-a797- src  # 放行到 0.0.0.0/0 匹配 ipset NIPv422b33e63-0f84-4bbc-a797- 的包 
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */  # 丢弃 INVALID 的包
    0     0 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */  # 将不匹配的通信发送到回退链
stack@xiexianbin-vm:~$ sudo ipset list NIPv422b33e63-0f84-4bbc-a797-
Name: NIPv422b33e63-0f84-4bbc-a797-
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 512
References: 1
Number of entries: 1
Members:
10.0.0.29  # ipset 匹配 10.0.0.29

Linux ipset 使用介绍

新建防火墙规则对 iptables 的影响

新建允许从 10.1.0.0/24 ping 主机的规则到 default 安全组,命令如下:

stack@xiexianbin-vm:~$ openstack security group rule create 22b33e63-0f84-4bbc-a797-7c3d7ae107aa --remote-ip 10.1.0.0/24 --protocol icmp --ingress
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2021-01-03T03:48:27Z                 |
| description       |                                      |
| direction         | ingress                              |
| ether_type        | IPv4                                 |
| id                | 3588e58b-1890-4929-ab85-d5519ec1b7a7 |
| name              | None                                 |
| port_range_max    | None                                 |
| port_range_min    | None                                 |
| project_id        | 45837e6267c44d7788a919d6e342e64c     |
| protocol          | icmp                                 |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 10.1.0.0/24                          |
| revision_number   | 0                                    |
| security_group_id | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa |
| tags              | []                                   |
| updated_at        | 2021-01-03T03:48:27Z                 |
+-------------------+--------------------------------------+

查看 iptables 规则,多了一条:

$ iptables -nvL
Chain neutron-openvswi-i33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     icmp --  *      *       10.1.0.0/24          0.0.0.0/0  # 放行从 `10.1.0.0/24` 发出的 icmp 包

至此,我们可以从 10.1.0.0/24 ping 通云主机,安全组规则生效。类似的出站规则也是这样工作。

allow_address_key_pairs

port 10.0.0.29 添加 allowed-address 0.0.0.0/0,操作如下:

stack@xiexianbin-vm:~$ openstack port set 33d3e592-c6eb-4cb3-8266-33c8e2e1f646 --allowed-address ip-address=0.0.0.0/0
stack@xiexianbin-vm:~$ openstack port show 33d3e592-c6eb-4cb3-8266-33c8e2e1f646
+-------------------------+-------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                       |
+-------------------------+-------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                          |
| allowed_address_pairs   | ip_address='0.0.0.0/0', mac_address='fa:16:3e:57:1c:cd'                                                     |
| binding_host_id         | xiexianbin-vm                                                                                               |
| binding_profile         |                                                                                                             |
| binding_vif_details     | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='True', port_filter='True' |
| binding_vif_type        | ovs                                                                                                         |
| binding_vnic_type       | normal                                                                                                      |
| created_at              | 2021-01-03T01:30:54Z                                                                                        |
| data_plane_status       | None                                                                                                        |
| description             |                                                                                                             |
| device_id               | a9afb91b-8700-4901-9565-5bf6a63f0143                                                                        |
| device_owner            | compute:nova                                                                                                |
| dns_assignment          | None                                                                                                        |
| dns_domain              | None                                                                                                        |
| dns_name                | None                                                                                                        |
| extra_dhcp_opts         |                                                                                                             |
| fixed_ips               | ip_address='10.0.0.29', subnet_id='3768efcb-e21e-4293-a1f9-596e28a21cb5'                                    |
| id                      | 33d3e592-c6eb-4cb3-8266-33c8e2e1f646                                                                        |
| ip_allocation           | None                                                                                                        |
| mac_address             | fa:16:3e:57:1c:cd                                                                                           |
| name                    |                                                                                                             |
| network_id              | 557bc377-cece-41dd-8b77-1348355ff66c                                                                        |
| numa_affinity_policy    | None                                                                                                        |
| port_security_enabled   | True                                                                                                        |
| project_id              | 45837e6267c44d7788a919d6e342e64c                                                                            |
| propagate_uplink_status | None                                                                                                        |
| qos_network_policy_id   | None                                                                                                        |
| qos_policy_id           | None                                                                                                        |
| resource_request        | None                                                                                                        |
| revision_number         | 5                                                                                                           |
| security_group_ids      | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa                                                                        |
| status                  | ACTIVE                                                                                                      |
| tags                    |                                                                                                             |
| trunk_details           | None                                                                                                        |
| updated_at              | 2021-01-03T03:56:17Z                                                                                        |
+-------------------------+-------------------------------------------------------------------------------------------------------------+

Chain neutron-openvswi-s33d3e592-c 增加一条对所有 0.0.0.0/0 FA:16:3E:57:1C:CD 地址来的流量放行的规则,如下:

Chain neutron-openvswi-s33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            MAC FA:16:3E:57:1C:CD /* Allow traffic from defined IP/MAC pairs. */

安全组规则没有发生变化(其他场景也可能新增 match-set,本示例已经存在),我们查看 ipset 信息如下:

stack@xiexianbin-vm:~$ sudo ipset list NIPv422b33e63-0f84-4bbc-a797-
Name: NIPv422b33e63-0f84-4bbc-a797-
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 640
References: 1
Number of entries: 3
Members:
128.0.0.0/1
0.0.0.0/1  # 放行所有流量
10.0.0.29

从 ipset 看,当添加 --allowed-address ip-address=0.0.0.0/0 时,会放行所有流量,导致安全组失效。

因此,我们建议在生产环境,添加 --allowed-address 时,尽量指定地址范围,避免安全组失效的问题。指定 --allowed-address ip-address=10.0.0.128/26 时,ipset 信息如下:

stack@xiexianbin-vm:~$ sudo ipset list NIPv422b33e63-0f84-4bbc-a797-
Name: NIPv422b33e63-0f84-4bbc-a797-
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 640
References: 1
Number of entries: 3
Members:
128.0.0.0/1
10.0.0.128/26  # 放行指定网段流量
10.0.0.29

导致安全组失效,主要是自身添加了到自身安全组放行导致的,我们删除对应的规则,如下:

stack@xiexianbin-vm:~$ openstack security group list
+--------------------------------------+---------+------------------------+----------------------------------+------+
| ID                                   | Name    | Description            | Project                          | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+
| 22b33e63-0f84-4bbc-a797-7c3d7ae107aa | default | Default security group | 45837e6267c44d7788a919d6e342e64c | []   |
+--------------------------------------+---------+------------------------+----------------------------------+------+
stack@xiexianbin-vm:~$ openstack security group rule list 22b33e63-0f84-4bbc-a797-7c3d7ae107aa
+--------------------------------------+-------------+-----------+-------------+------------+--------------------------------------+
| ID                                   | IP Protocol | Ethertype | IP Range    | Port Range | Remote Security Group                |
+--------------------------------------+-------------+-----------+-------------+------------+--------------------------------------+
| 09c623fc-bf1f-46bf-9149-facdc6148831 | None        | IPv6      | ::/0        |            | None                                 |
| 0f709d8c-7400-40dd-a5b7-59beaee93c74 | None        | IPv6      | ::/0        |            | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa |
| 3588e58b-1890-4929-ab85-d5519ec1b7a7 | icmp        | IPv4      | 10.1.0.0/24 |            | None                                 |
| 366c1188-988b-4c5a-8bf4-dfcec6fb902f | None        | IPv4      | 0.0.0.0/0   |            | 22b33e63-0f84-4bbc-a797-7c3d7ae107aa |
| 6ff01e6d-9b34-433c-968f-c04495098145 | None        | IPv4      | 0.0.0.0/0   |            | None                                 |
+--------------------------------------+-------------+-----------+-------------+------------+--------------------------------------+
stack@xiexianbin-vm:~$ openstack security group rule delete 0f709d8c-7400-40dd-a5b7-59beaee93c74 366c1188-988b-4c5a-8bf4-dfcec6fb902f
stack@xiexianbin-vm:~$ openstack security group rule list 22b33e63-0f84-4bbc-a797-7c3d7ae107aa
+--------------------------------------+-------------+-----------+-------------+------------+-----------------------+
| ID                                   | IP Protocol | Ethertype | IP Range    | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+-------------+------------+-----------------------+
| 09c623fc-bf1f-46bf-9149-facdc6148831 | None        | IPv6      | ::/0        |            | None                  |
| 3588e58b-1890-4929-ab85-d5519ec1b7a7 | icmp        | IPv4      | 10.1.0.0/24 |            | None                  |
| 6ff01e6d-9b34-433c-968f-c04495098145 | None        | IPv4      | 0.0.0.0/0   |            | None                  |
+--------------------------------------+-------------+-----------+-------------+------------+-----------------------+

iptables Chain neutron-openvswi-i33d3e592-c 对应的 ipset NIPv422b33e63-0f84-4bbc-a797- 也被删除,因此安全组可以正常工作,如下:

stack@xiexianbin-vm:~$ sudo iptables -nvL neutron-openvswi-i33d3e592-c
Chain neutron-openvswi-i33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination
    5   420 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
    2   696 RETURN     udp  --  *      *       0.0.0.0/0            10.0.0.29            udp spt:67 dpt:68
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            255.255.255.255      udp spt:67 dpt:68
    0     0 RETURN     icmp --  *      *       10.1.0.0/24          0.0.0.0/0
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
    0     0 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */

在管理其他安全组,观察ipset变化,如下:

stack@xiexianbin-vm:~$ openstack security group create demo-1
+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                                                                                 |
+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at      | 2021-01-03T05:29:25Z                                                                                                                                  |
| description     | demo-1                                                                                                                                                |
| id              | 178a2ad5-f758-4af0-a920-1e54cc8260e5                                                                                                                  |
| name            | demo-1                                                                                                                                                |
| project_id      | 45837e6267c44d7788a919d6e342e64c                                                                                                                      |
| revision_number | 1                                                                                                                                                     |
| rules           | created_at='2021-01-03T05:29:25Z', direction='egress', ethertype='IPv6', id='26d6494c-4bf5-44c1-91d4-0525c32046f5', updated_at='2021-01-03T05:29:25Z' |
|                 | created_at='2021-01-03T05:29:25Z', direction='egress', ethertype='IPv4', id='7a48bb5c-319d-4963-a7bc-406729e27a78', updated_at='2021-01-03T05:29:25Z' |
| stateful        | True                                                                                                                                                  |
| tags            | []                                                                                                                                                    |
| updated_at      | 2021-01-03T05:29:25Z                                                                                                                                  |
+-----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+

iptables Chain neutron-openvswi-i33d3e592-c 对应的 ipset NIPv4178a2ad5-f758-4af0-a920- members 为空,安全组工作正常。

stack@xiexianbin-vm:~$ sudo iptables -nvL neutron-openvswi-i33d3e592-c
Chain neutron-openvswi-i33d3e592-c (1 references)
 pkts bytes target     prot opt in     out     source               destination
    5   420 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
    2   696 RETURN     udp  --  *      *       0.0.0.0/0            10.0.0.29            udp spt:67 dpt:68
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            255.255.255.255      udp spt:67 dpt:68
    0     0 RETURN     icmp --  *      *       10.1.0.0/24          0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set NIPv4178a2ad5-f758-4af0-a920- src
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
    0     0 neutron-openvswi-sg-fallback  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Send unmatched traffic to the fallback chain. */
stack@xiexianbin-vm:~$ sudo ipset list NIPv4178a2ad5-f758-4af0-a920-
Name: NIPv4178a2ad5-f758-4af0-a920-
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 448
References: 1
Number of entries: 0
Members:

https://www.xiexianbin.cn/openstack/neutron/2015-03-21-neutron-havana-allow-address-pairs/index.html

专栏文章
  1. 使用 Devstack 搭建 Openstack 开发环境
  2. OpenStack 启动第一个云主机
  3. Neutron 基于 iptables 实现安全组功能(当前)
  4. KVM 网络虚拟化基础
  5. CPU 和内存虚拟化原理
  6. 虚拟化技术简介
  7. cloudbase-init 安装使用与配置
  8. 下一代云平台强调“整合”
  9. openstack中的metadata server
最新评论
加载中...