在Devstack中使能Cinder的Ceph RBD driver

版权声明：本文为博主原创文章，可以转载但必须注明出处。 https://blog.csdn.net/nirendao/article/details/84675140

假设已经有了一个DevStack和一个Ceph集群，如何使DevStack上的Cinder能用上Ceph作为其存储后端呢？
以下步骤如未做特殊说明，都是在DevStack虚机上操作。

1. 使该DevStack VM变成一个Ceph client

1.1 安装 ceph-common

yum install ceph-common  # 或者
apt install ceph-common

1.2 拷贝 Ceph Cluster 的ceph.conf 和 keyring 到 DevStack VM 的 /etc/ceph

# On DevStack
sudo mkdir /etc/ceph 
sudo -R stack:stack /etc/ceph    # 这一步很重要，否则RBD driver不能工作

# On monitor node of Ceph cluster 
scp /etc/ceph/ceph.conf <user>@<devstack_vm>:/etc/ceph
scp /etc/ceph/xxx.keyring <user>@<devstack_vm>:/etc/ceph

2. 修改/etc/cinder/cinder.conf

2.1 在最后添加如下一段：

[ceph-rbd]
volume_backend_name=ceph-rbd
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_user=admin
rbd_pool=rbd

2.2 在enabled_backends一行加上自定义的volume_backend_name,即加上ceph-rbd

enabled_backends = lvmdriver-1,ceph-rbd

3. 重启所有 cinder服务

sudo systemctl restart devstack@c-*   # c-api, c-sch, c-vol

4. 添加新driver类型

cd /opt/stack/devstack
source openrc admin admin
openstack volume type create ceph-rbd
openstack volume type set --property volume_backend_name=ceph-rbd ceph-rbd
cinder type-list  # validate the drivers
cinder extra-specs-list  # validate the backends

5. 测试

cinder create --name cinder_disk_1 --volume-type ceph-rbd  10
cinder list

6. 其他

如果出错了，查看日志的方法

sudo journalctl -f -u devstack@c-vol

7. 使用程序调试

最后，附一段Python代码可用来调试RBD driver到Ceph cluster的连通性。

import rados
import rbd

def connect_to_rados(user, name='ceph', pool='rbd',
                     conf='/etc/ceph/ceph.conf'):
    client = rados.Rados(rados_id=user, 
                         clustername=name, 
                         conffile=conf)
    try:
        client.connect()
        ioctx = client.open_ioctx(pool)
        return client, ioctx
    except rados.Error:
        msg = "Error connecting to ceph cluster."
        print(msg)
        client.shutdown()
    

def create_rbd(client, ioctx, name):
    rbd_inst = rbd.RBD()
    size = 1 * 1024**3
    rbd_inst.create(ioctx, name, size)
    
    
def delete_rbd(ioctx, rbd_name):
    rbd_inst = rbd.RBD()
    rbd_inst.remove(ioctx, rbd_name)
    
def main():
    client, ioctx = connect_to_rados('admin')
    rbd_name = 'finix_rbd_01'
    create_rbd(client, ioctx, rbd_name)
    # delete_rbd(ioctx, rbd_name)
    
    
main()