客户端使用ceph服务器的块设备

环境:
Os:Centos 7
[root@master /]# ceph version
ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable)

 

#############################管理节点上前置事项####################
1.在管理节点上,将 ceph-client 添加到管理节点的 hosts 文件
echo '192.168.1.111 ceph-client' >> /etc/hosts

 

2.在管理节点上,允许管理节点无密码登录 ceph-client
ssh-copy-id root@ceph-client

 

3.在管理节点上,通过 ceph-deploy 把 Ceph 安装到 ceph-client 节点
[root@master ceph]#cd /opt/ceph
[root@master ceph]#ceph-deploy install ceph-client

 

4.在管理节点上,用 ceph-deploy 把 Ceph 配置文件和 ceph.client.admin.keyring 拷贝到 ceph-client 。
[root@master ceph]#ceph-deploy admin ceph-client

 

#########################客户端的操作####################
1.创建一个pool
[root@master /]# ceph osd pool create hxl_pool_test 8 8
pool 'hxl_pool_test' created

 

2.查看创建的pool

[root@master /]# ceph osd pool ls detail
pool 2 'dynamics-pool' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 107 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbdremoved_snaps [1~3]
pool 3 'hxl_pool_test' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 110 flags hashpspool stripe_width 0

 

3.首先先创建一个块设备

[root@master /]# rbd create hxl_pool_test/myrbd1 --size=100MB --image-feature layering
用法:
rbd create (pool_name)/（rbd_name） --size xxxxxMB就可以
查看pool下的块设备
[root@master /]# rbd -p hxl_pool_test ls
myrbd1

 

[root@master /]# rbd info hxl_pool_test/myrbd1 
rbd image 'myrbd1':size 100 MiB in 25 objectsorder 22 (4 MiB objects)snapshot_count: 0id: 4f333b4d4de0eblock_name_prefix: rbd_data.4f333b4d4de0eformat: 2features: layeringop_features: flags: create_timestamp: Mon Oct 20 15:34:03 2025access_timestamp: Mon Oct 20 15:34:03 2025modify_timestamp: Mon Oct 20 15:34:03 2025

 

4.映射块设备到你的机器

[root@master /]# rbd map hxl_pool_test/myrbd1
/dev/rbd0[root@master /]#  rbd showmapped
id pool          namespace image  snap device    
0  hxl_pool_test           myrbd1 -    /dev/rbd0 

 

5.错误解决(application not enabled on 1 pool)

[root@master /]#ceph -s
[root@master /]#ceph health detailcluster:id:     1508a2da-5991-487a-836c-d6e6527b1dc7health: HEALTH_WARNapplication not enabled on 1 pool(s)
执行如下命令:
[root@master /]#ceph osd pool application enable hxl_pool_test rbd

 

6.挂载并使用

[root@master ceph]# mkfs.xfs /dev/rbd0
[root@master ceph]# mount /dev/rbd0 /mnt
[root@ceph-1 ceph]# df -h
/dev/rbd0               1014M   33M  982M   4% /mnt

 

7.扩容

[root@master /]# rbd resize --size 200MB hxl_pool_test/myrbd1
Resizing image: 100% complete...done.[root@master /]# rbd info hxl_pool_test/myrbd1
rbd image 'myrbd1':size 200 MiB in 50 objectsorder 22 (4 MiB objects)snapshot_count: 0id: 4f333b4d4de0eblock_name_prefix: rbd_data.4f333b4d4de0eformat: 2features: layeringop_features: flags: create_timestamp: Mon Oct 20 15:34:03 2025access_timestamp: Mon Oct 20 15:34:03 2025modify_timestamp: Mon Oct 20 15:34:03 2025

 

##支持文件系统在线扩容

[root@master /]# resize2fs /dev/rbd0
resize2fs 1.42.9 (28-Dec-2013)
resize2fs: Bad magic number in super-block while trying to open /dev/rbd0
Couldn't find valid filesystem superblock.


解决办法:
[root@master /]# xfs_growfs /dev/rbd0
meta-data=/dev/rbd0              isize=512    agcount=4, agsize=7168 blks=                       sectsz=512   attr=2, projid32bit=1=                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=25600, imaxpct=25=                       sunit=1024   swidth=1024 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=624, version=2=                       sectsz=512   sunit=8 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 25600 to 51200[root@master /]# 
/dev/rbd0                198M  5.5M  193M   3% /mnt

 

8.解挂文件系统

[root@master mnt]# umount /mnt
umount: /mnt: target is busy.(In some cases useful info about processes that usethe device is found by lsof(8) or fuser(1))
[root@master mnt]# fuser -m /mnt
[root@master mnt]# kill -9 PID

 

9.取消块设备映射
[root@master /]# rbd unmap hxl_pool_test/myrbd1
[root@master /]# rbd showmapped

10.删除块设备
[root@master /]# rbd rm hxl_pool_test/myrbd1
Removing image: 100% complete...done.

[root@master /]# rbd ls hxl_pool_test