本文介绍基于
devstack搭建的OpenStack环境创建云主机
项目、用户创建
$ openstack domain create --description "Default Domain" default
$ openstack project create --domain default --description "Admin Project" admin
$ openstack user create --domain default --password-prompt admin
$ openstack role create admin
$ openstack role add --project admin --user admin admin
创建cirros镜像
下载地址：http://download.cirros-cloud.net/
默认用户名/密码：cirros/gocubsgo
openstack image create cirros-0.5.1-x86_64-disk --public --container-format bare --disk-format qcow2 admin < cirros-0.5.x-aarch64-disk.img
网络创建
创建network
openstack-network-create ↕
$ openstack network create vxlan-1 --provider-network-type vxlan --provider-segment 100 --internal --project admin --enable --no-share
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2020-01-03T09:28:28Z |
| description | |
| dns_domain | None |
| id | e022f4ea-e955-4bcd-b2fa-af6616d3eb12 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1450 |
| name | vxlan-1 |
| port_security_enabled | True |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| provider:network_type | vxlan |
| provider:physical_network | None |
| provider:segmentation_id | 100 |
| qos_policy_id | None |
| revision_number | 1 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | 2020-01-03T09:28:28Z |
+---------------------------+--------------------------------------+
$ openstack network list
+--------------------------------------+---------+---------+
| ID | Name | Subnets |
+--------------------------------------+---------+---------+
| e022f4ea-e955-4bcd-b2fa-af6616d3eb12 | vxlan-1 | |
+--------------------------------------+---------+————+
创建subnet
openstack-subnet-create ↕
$ openstack subnet create vxlan-subnet-1 --project admin --subnet-range 10.0.0.0/24 --dhcp --gateway 10.0.0.1 --ip-version 4 --network vxlan-1
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.0.0.2-10.0.0.254 |
| cidr | 10.0.0.0/24 |
| created_at | 2020-01-03T09:31:37Z |
| description | |
| dns_nameservers | |
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 10.0.0.1 |
| host_routes | |
| id | 919e1444-ae5f-45da-981e-1140fbcaa246 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | vxlan-subnet-1 |
| network_id | e022f4ea-e955-4bcd-b2fa-af6616d3eb12 |
| prefix_length | None |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-01-03T09:31:37Z |
+----------------------+--------------------------------------+
$ openstack subnet list
+--------------------------------------+----------------+--------------------------------------+-------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------+--------------------------------------+-------------+
| 919e1444-ae5f-45da-981e-1140fbcaa246 | vxlan-subnet-1 | e022f4ea-e955-4bcd-b2fa-af6616d3eb12 | 10.0.0.0/24 |
+--------------------------------------+----------------+--------------------------------------+-------------+
$ openstack port list --network e022f4ea-e955-4bcd-b2fa-af6616d3eb12
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+
| 9f56395b-d1ef-44d4-8ec0-7ba3970177fc | | fa:16:3e:4e:32:1c | ip_address='10.0.0.2', subnet_id='919e1444-ae5f-45da-981e-1140fbcaa246' | ACTIVE |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------+--------+
创建子网时：
- 若指定
--dhcp会在网络节点自动创建Linux net namespaces，netns的名称为qdhcp-<network-id>。其中10.0.0.2为dhcp的IP地址。 - 若没有指定
--dhcp，当前不会创建netns
查看 netns 中信息如下：
ovs-netns-1 ↕
$ sudo ip netns
qdhcp-e022f4ea-e955-4bcd-b2fa-af6616d3eb12 (id: 0)
$ sudo ip netns exec qdhcp-e022f4ea-e955-4bcd-b2fa-af6616d3eb12 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
10: tap9f56395b-d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:4e:32:1c brd ff:ff:ff:ff:ff:ff
inet 10.0.0.2/24 brd 10.0.0.255 scope global tap9f56395b-d1
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe4e:321c/64 scope link
valid_lft forever preferred_lft forever
$ sudo ip netns exec qdhcp-e022f4ea-e955-4bcd-b2fa-af6616d3eb12 ip r
default via 10.0.0.1 dev tap9f56395b-d1 proto static
10.0.0.0/24 dev tap9f56395b-d1 proto kernel scope link src 10.0.0.2
可以在 netns 看到一个tap设备：tap9f56395b-d1，IP地址为：10.0.0.2/24
- tap设备命名规则：
tap<port-id-prefix-11> tap9f56395b-d1会自动挂载到OVS的br-int上
ovs-vsctl-1 ↕
$ sudo ovs-vsctl show
0976470f-8041-45d1-ad1a-34cd061684d3
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-int
Interface br-int
type: internal
Port tap9f56395b-d1
tag: 1
Interface tap9f56395b-d1
type: internal
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
ovs_version: "2.13.1"
创建安全组
openstack-security-group-create ↕
$ openstack security group create default --project admin
$ openstack security group show 6e28e3c1-6959-4dfb-99b1-b7277e6353a0
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2021-01-03T06:23:51Z |
| description | Default security group |
| id | 6e28e3c1-6959-4dfb-99b1-b7277e6353a0 |
| name | default |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| revision_number | 3 |
| rules | created_at='2021-01-03T06:23:51Z', direction='egress', ethertype='IPv4', id='25d6afb9-3b31-408f-b71c-f14c20287ca5', updated_at='2021-01-03T06:23:51Z' |
| | created_at='2021-01-03T06:23:51Z', direction='ingress', ethertype='IPv6', id='674d9013-7ad3-4145-bec2-b351b896d479', remote_group_id='6e28e3c1-6959-4dfb-99b1-b7277e6353a0', updated_at='2021-01-03T06:23:51Z' |
| | created_at='2021-01-03T06:23:51Z', direction='egress', ethertype='IPv6', id='c1333e8e-eb4c-44fd-818f-26753db655a9', updated_at='2021-01-03T06:23:51Z' |
| | created_at='2021-01-03T06:23:51Z', direction='ingress', ethertype='IPv4', id='de36561f-8f73-40b6-b78f-ea06cc4307ae', remote_group_id='6e28e3c1-6959-4dfb-99b1-b7277e6353a0', updated_at='2021-01-03T06:23:51Z' |
| stateful | True |
| tags | [] |
| updated_at | 2021-01-03T02:32:49Z |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
可以根据自己的需求，添加其他的规则。
创建vm
创建
openstack-server-create ↕
$ openstack server create vm-1 --image cirros-0.5.1-x86_64-disk --flavor m1.tiny --security-group 6e28e3c1-6959-4dfb-99b1-b7277e6353a0 --network vxlan-1
+-------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | B7KGjrUZP82D |
| config_drive | |
| created | 2020-01-03T09:54:57Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | bd023f89-a94b-44a0-b119-712cca9da61f |
| image | cirros-0.5.1-x86_64-disk (000fcfa3-7ffa-4125-817e-cf7a27cf3eb1) |
| key_name | None |
| name | vm-1 |
| progress | 0 |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| properties | |
| security_groups | name='6e28e3c1-6959-4dfb-99b1-b7277e6353a0' |
| status | BUILD |
| updated | 2020-01-03T09:54:56Z |
| user_id | c6b926ebfb5c4a6db670150b183581af |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------+
$ openstack server list
+--------------------------------------+------+--------+--------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+--------------------+--------------------------+---------+
| bd023f89-a94b-44a0-b119-712cca9da61f | vm-1 | ACTIVE | vxlan-1=10.0.0.242 | cirros-0.5.1-x86_64-disk | m1.tiny |
+--------------------------------------+------+--------+--------------------+--------------------------+---------+
$ openstack port list
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+
| 0a82ef18-9834-4baf-9f96-c2a586e56f48 | | fa:16:3e:34:27:00 | ip_address='10.0.0.242', subnet_id='919e1444-ae5f-45da-981e-1140fbcaa246' | ACTIVE |
| 9f56395b-d1ef-44d4-8ec0-7ba3970177fc | | fa:16:3e:4e:32:1c | ip_address='10.0.0.2', subnet_id='919e1444-ae5f-45da-981e-1140fbcaa246' | ACTIVE |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+
$ openstack port show 0a82ef18-9834-4baf-9f96-c2a586e56f48
+-------------------------+--------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | xiexianbin-vm |
| binding_profile | |
| binding_vif_details | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2020-01-03T09:54:59Z |
| data_plane_status | None |
| description | |
| device_id | bd023f89-a94b-44a0-b119-712cca9da61f |
| device_owner | compute:nova |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.0.0.242', subnet_id='919e1444-ae5f-45da-981e-1140fbcaa246' |
| id | 0a82ef18-9834-4baf-9f96-c2a586e56f48 |
| ip_allocation | None |
| mac_address | fa:16:3e:34:27:00 |
| name | |
| network_id | e022f4ea-e955-4bcd-b2fa-af6616d3eb12 |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 5 |
| security_group_ids | 6e28e3c1-6959-4dfb-99b1-b7277e6353a0 |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2020-01-03T10:04:02Z |
+-------------------------+--------------------------------------------------------------------------------------------------------------+
新创建主机的IP地址为：10.0.0.242
qemu 进程
qemu-proc ↕
$ ps -ef| grep qemu
libvirt+ 4221 1 1 09:35 ? 00:00:23 /usr/bin/qemu-system-x86_64 -name guest=instance-00000001,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-instance-00000001/master-key.aes -machine pc-i440fx-4.2,accel=tcg,usb=off,dump-guest-core=off -cpu qemu64 -m 512 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -uuid bd023f89-a94b-44a0-b119-712cca9da61f -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=22.2.1,serial=bd023f89-a94b-44a0-b119-712cca9da61f,uuid=bd023f89-a94b-44a0-b119-712cca9da61f,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=31,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -blockdev {"driver":"file","filename":"/opt/stack/data/nova/instances/_base/50baf97670ca5516043b3c4de1dec8d12fb1449c","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":true,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"} -blockdev {"driver":"file","filename":"/opt/stack/data/nova/instances/bd023f89-a94b-44a0-b119-712cca9da61f/disk","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"} -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on -netdev tap,fd=33,id=hostnet0 -device virtio-net-pci,host_mtu=1450,netdev=hostnet0,id=net0,mac=fa:16:3e:34:27:00,bus=pci.0,addr=0x3 -add-fd set=2,fd=35 -chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on -device isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
其中，网络配置 tap,fd=33，信息如下：
-netdev tap,fd=33,id=hostnet0 -device virtio-net-pci,host_mtu=1450,netdev=hostnet0,id=net0,mac=fa:16:3e:34:27:00,bus=pci.0,addr=0x3
查看 qemu 的 /proc/4221/fd/33 指向 /dev/net/tun 设备
$ sudo ls -lh /proc/4221/fd/ |grep tun
lrwx------ 1 libvirt-qemu kvm 64 Mar 24 10:13 33 -> /dev/net/tun
$ sudo file /proc/4221/fd/33
/proc/4221/fd/33: symbolic link to /dev/net/tun
$ sudo cat /proc/4221/fdinfo/33
pos: 54
flags: 0104002
mnt_id: 27
iff: tap0a82ef18-98 # 与OVS br-int的名称一致
其他查看 tap/tun 设备命令：
$ sudo ip tuntap list | grep tap0a82ef18-98
tap0a82ef18-98: tap vnet_hdr
OVS 信息
- ovs br 信息
ovs-vsctl-2 ↕
$ sudo ovs-vsctl show
0976470f-8041-45d1-ad1a-34cd061684d3
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
datapath_type: system
Port br-int
Interface br-int
type: internal
Port tap9f56395b-d1
tag: 1
Interface tap9f56395b-d1
type: internal
Port tap0a82ef18-98
tag: 1
Interface tap0a82ef18-98
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
ovs_version: "2.13.1"
其中 tap0a82ef18-98 为新增的 tap设备，0a82ef18-98 为 port 10.0.0.242 ID 0a82ef18-9834-4baf-9f96-c2a586e56f48 的前11位
因此，VM 在 OVS br-int 网桥 tap 命令规则为：tap<port-id-prefix-11>
- ovs br-int 流表信息
ovs-ofctl-1 ↕
$ sudo ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000a62b2fabc24d
n_tables:254, n_buffers:0
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
1(int-br-ex): addr:be:8b:a6:4f:16:9e
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
2(patch-tun): addr:d2:54:6c:cf:6a:93
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
3(tap9f56395b-d1): addr:fa:16:3e:4e:32:1c
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
5(tap0a82ef18-98): addr:fe:16:3e:34:27:00
config: 0
state: 0
current: 10MB-FD COPPER
speed: 10 Mbps now, 0 Mbps max
LOCAL(br-int): addr:a6:2b:2f:ab:c2:4d
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
ping 验证
- 创建安全组
$ openstack security group rule create 6e28e3c1-6959-4dfb-99b1-b7277e6353a0 --ingress --remote-ip 0.0.0.0/0 --protocol icmp --project 45837e6267c44d7788a919d6e342e64c
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| created_at | 2021-01-03T02:39:06Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 4f067654-818a-4d90-b0d6-d0ffc346e0a4 |
| name | None |
| port_range_max | None |
| port_range_min | None |
| project_id | 45837e6267c44d7788a919d6e342e64c |
| protocol | icmp |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | 0 |
| security_group_id | 6e28e3c1-6959-4dfb-99b1-b7277e6353a0 |
| tags | [] |
| updated_at | 2021-01-03T02:39:06Z |
+-------------------+--------------------------------------+
- 在
netns中ping
$ sudo ip netns exec qdhcp-e022f4ea-e955-4bcd-b2fa-af6616d3eb12 ping -c 1 10.0.0.242
PING 10.0.0.242 (10.0.0.242) 56(84) bytes of data.
64 bytes from 10.0.0.242: icmp_seq=1 ttl=64 time=1.17 ms
--- 10.0.0.242 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.178/1.178/1.178/0.000 ms
至此，基于OpenStack创建VM已经介绍完毕，接下来会依次介绍VM其他配置。