CentOS 6.6 默认安装的 libvirt-0.10.2 是不支持 storage-rbd，若要支持 Ceph Pool 功能，需要重新编译 libvirt。
重新编译 libvirt
CentOS 6.6 的 src.rpm 包下载 libvirt-0.10.2-46.el6_6.6.src.rpm，然后执行编译：
rpmbuild --rebuild libvirt-0.10.2-46.el6_6.6.src.rpm
然后提示一堆需要的依赖包，自己安装进去。当然不是直接编译这个 libvirt-0.10.2-46.el6_6.6.src.rpm。巧妙之处是，在编译过程中，Ctrl+c终止这个过程，然后修改配置文件
/root/rpmbuild/SPECS/libvirt.spec
79 %if 0%{?fedora} >= 16 `|| 0%{?centos} >= 6`
80 %define with_storage_rbd 0%{!?_without_storage_rbd:%{server_drivers}}
81 %else
82 %define with_storage_rbd 0
83 %endif
需要增加|| 0%{?centos} >= 6。OK，好，可以继续编译，编译命令是
rpmbuild --bb /root/rpmbuild/SPECS/libvirt.spec
不过又会遇到问题，你需要先安装ceph-devel包。后面继续报错，就是storage_backend_rbd.c里面定义了一个结构state，这个和 include 进来的头文件冲突，解决方法是把storage_backend_rbd.c定义那个:
state
更改为:
clusterstate
好，这样在执行
rpmbuild --bb /root/rpmbuild/SPECS/libvirt.spec
应该没有问题了。生成文件：
[root@ceph-libvirt x86_64]# ll
total 12012
-rw-r--r--. 1 root root 2928971 May 27 16:15 libvirt-0.10.2-46.0.1.el6.6.x86_64.rpm
-rw-r--r--. 1 root root 7256817 May 27 16:15 libvirt-client-0.10.2-46.0.1.el6.6.x86_64.rpm
-rw-r--r--. 1 root root 1044981 May 27 16:15 libvirt-devel-0.10.2-46.0.1.el6.6.x86_64.rpm
-rw-r--r--. 1 root root 275625 May 27 16:15 libvirt-lock-sanlock-0.10.2-46.0.1.el6.6.x86_64.rpm
-rw-r--r--. 1 root root 781811 May 27 16:15 libvirt-python-0.10.2-46.0.1.el6.6.x86_64.rpm
然后就是对接 Ceph/RBD。先创建一个 Pool：
[root@ceph-mon-1 libvirtd]# ceph osd pool create storage1 128 128
pool 'storage1' created
[root@ceph-mon-1 libvirtd]# ceph auth get-or-create client.storage1 mon 'allow r' osd 'allow rwx pool=storage1'
[client.storage1]
key = AQD4gWVVSFNRHRAA4WOd8unlNCxnbgPhBIhTlA==
[root@ceph-mon-1 libvirtd]#
这里创建了 Pool 并且生成了一个访问凭证。在计算节点编辑文件secret-storage1.xml：
<secret ephemeral='no' private='no'>
<usage type='ceph'>
<name>storage1</name>
</usage>
</secret>
然后就是导入：
[root@ceph-libvirt ceph]# virsh secret-define --file secret-storage1.xml
Secret de3ea4e8-fb19-9dfb-fe01-b0de5aa172fd created
然后就是填充密钥：
[root@ceph-libvirt ceph]# virsh
Welcome to virsh, the virtualization interactive terminal.
Type: 'help' for help with commands
'quit' to quit
virsh # secret-list
UUID Usage
de3ea4e8-fb19-9dfb-fe01-b0de5aa172fd Unused
virsh # secret-set-value de3ea4e8-fb19-9dfb-fe01-b0de5aa172fd AQD4gWVVSFNRHRAA4WOd8unlNCxnbgPhBIhTlA==
Secret value set
virsh #
编译一个 pool 文件 storage1.xml：
<pool type='rbd'>
<name>storage1</name>
<source>
<host name='10.89.13.71' port='6789'/>
<name>storage1</name>
<auth type='ceph' username='storage1'>
<secret uuid='de3ea4e8-fb19-9dfb-fe01-b0de5aa172fd'/>
</auth>
</source>
</pool>
然后定义这个 pool 文件：
virsh pool-define storage1.xml
[root@ceph-libvirt ceph]# virsh
Welcome to virsh, the virtualization interactive terminal.
Type: 'help' for help with commands
'quit' to quit
virsh # pool-list --all
Name State Autostart
storage1 inactive no
virsh # pool-
pool-autostart pool-create-as pool-delete pool-edit pool-name pool-undefine
pool-build pool-define pool-destroy pool-info pool-refresh pool-uuid
pool-create pool-define-as pool-dumpxml pool-list pool-start
virsh # pool-start storage1
Pool storage1 started
virsh # pool-list --all
Name State Autostart
storage1 active no
virsh # pool-dumpxml storage1
<pool type='rbd'>
<name>storage1</name>
<uuid>107550ae-3d6c-38de-9a78-55069ec62d03</uuid>
<capacity unit='bytes'>1173478178816</capacity>
<allocation unit='bytes'>0</allocation>
<available unit='bytes'>1170975944704</available>
<source>
<host name='10.89.13.71' port='6789'/>
<name>storage1</name>
<auth username='storage1' type='ceph'>
<secret uuid='de3ea4e8-fb19-9dfb-fe01-b0de5aa172fd'/>
</auth>
</source>
</pool>
咔咔，泄露了容量信息 .>_<. 然后创建一个块存储：
[root@ceph-libvirt ceph]# virsh
Welcome to virsh, the virtualization interactive terminal.
Type: 'help' for help with commands
'quit' to quit
virsh # vol-create-as --pool 107550ae-3d6c-38de-9a78-55069ec62d03 --name image1 --capacity 10G --backing-vol-format 2
virsh # vol-list --pool 107550ae-3d6c-38de-9a78-55069ec62d03
Name Path
image1 storage1/image1
virsh # vol-dumpxml --pool 107550ae-3d6c-38de-9a78-55069ec62d03 image1
<volume>
<name>image1</name>
<key>storage1/image1</key>
<source>
</source>
<capacity unit='bytes'>10737418240</capacity>
<allocation unit='bytes'>10737418240</allocation>
<target>
<path>storage1/image1</path>
<format type='unknown'/>
<permissions>
<mode>0600</mode>
<owner>4294967295</owner>
<group>4294967295</group>
</permissions>
</target>
</volume>
virsh #
看看 Ceph 那边：
[root@ceph-mon-1 libvirtd]# rbd ls storage1
image1
[root@ceph-mon-1 libvirtd]# rbd info storage1/image1
rbd image 'image1':
size 10240 MB in 2560 objects
order 22 (4096 kB objects)
block_name_prefix: rb.0.224fbb.6b12cdbd
format: 1
聪明的 cepher 估计已经看到一些瑕疵地方，估计在新版本会修复吧。