OpenStack(T版)——对象存储(Swift)服务介绍与安装

企业开发 2023-09-08 17:20:19 阅读次数: 0

文章目录

OpenStack(T版)——对象存储(Swift)服务介绍与安装

OpenStack 对象存储（Swift）是一种用于存储和管理大量数据的系统。它就像是一个超级大的云盘，可以存储各种各样的文件，比如照片、视频、文档等等。 与传统的文件存储不同，对象存储不关心文件的目录结构或层级关系，而是将每个文件视为一个独立的对象（Object）。每个对象都有一个唯一的标识符，我们可以通过这个标识符来找到和访问对象。

为了确保数据的可靠性和安全性，对象存储会将每个对象分成多个副本，并将这些副本分散存储在不同的存储服务器上。这样，即使某个服务器出现故障，我们仍然可以通过其他副本来获取文件。

当我们想要上传文件时，我们将文件分成小块，并将这些块分发到不同的存储服务器上。每个块都有自己的校验码，以便在需要时进行数据校验和修复。

当我们需要访问文件时，我们只需要提供文件的唯一标识符，对象存储系统就会根据标识符找到对应的存储服务器，并将文件块重新组装起来，然后将完整的文件返回给我们。

安装和配置(controller)

官方文档

准备

(1)加载admin user 的环境变量

[root@controller ~]# source admin-openrc.sh

(2)创建Identity服务凭据

①创建Swift用户

[root@controller ~]# openstack user create --domain default --password-prompt swift
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | 73688e8378464c28b658229ec741f597 |
| name                | swift                            |
| options             | {
    
    }                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+

②将admin role 赋予 swift user 和service project

[root@controller ~]#  openstack role add --project service --user swift admin

③创建 swift service entity

[root@controller ~]# openstack service create --name swift  --description "OpenStack Object Storage" object-store

+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Object Storage         |
| enabled     | True                             |
| id          | 232187105b6e4bc8b7dd4d8fb430beb8 |
| name        | swift                            |
| type        | object-store                     |
+-------------+----------------------------------+

(3)创建Swift对象存储服务组件的API endpouint

[root@controller ~]# openstack endpoint create --region RegionOne  object-store public http://controller:8080/v1/AUTH_%\(project_id\)s
+--------------+-----------------------------------------------+
| Field        | Value                                         |
+--------------+-----------------------------------------------+
| enabled      | True                                          |
| id           | 2ae6d8b8091a413ea11204fadd869296              |
| interface    | public                                        |
| region       | RegionOne                                     |
| region_id    | RegionOne                                     |
| service_id   | 232187105b6e4bc8b7dd4d8fb430beb8              |
| service_name | swift                                         |
| service_type | object-store                                  |
| url          | http://controller:8080/v1/AUTH_%(project_id)s |
+--------------+-----------------------------------------------+

[root@controller ~]# openstack endpoint create --region RegionOne  object-store internal http://controller:8080/v1/AUTH_%\(project_id\)s
+--------------+-----------------------------------------------+
| Field        | Value                                         |
+--------------+-----------------------------------------------+
| enabled      | True                                          |
| id           | 78f84dd843564cbebff06b52358811e9              |
| interface    | internal                                      |
| region       | RegionOne                                     |
| region_id    | RegionOne                                     |
| service_id   | 232187105b6e4bc8b7dd4d8fb430beb8              |
| service_name | swift                                         |
| service_type | object-store                                  |
| url          | http://controller:8080/v1/AUTH_%(project_id)s |
+--------------+-----------------------------------------------+

[root@controller ~]# openstack endpoint create --region RegionOne  object-store admin http://controller:8080/v1
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 910643f5738b4c35937bdcb938205f14 |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | 232187105b6e4bc8b7dd4d8fb430beb8 |
| service_name | swift                            |
| service_type | object-store                     |
| url          | http://controller:8080/v1        |
+--------------+----------------------------------+

安装和配置Swift对象存储服务组件

(1)安装软件包

[root@controller ~]# yum install -y openstack-swift-proxy python-swiftclient  python-keystoneclient python-keystonemiddleware  memcached

(2)Swift 代理服务器的配置文件/etc/swift/proxy-server.conf

[root@controller ~]# vim  /etc/swift/proxy-server.conf
[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift
[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server
[app:proxy-server]
use = egg:swift#proxy
account_autocreate = True
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = swift
password = 000000
delay_auth_decision = True
[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3
user_test5_tester5 = testing5 service
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = controller:11211
[filter:ratelimit]
use = egg:swift#ratelimit
[filter:domain_remap]
use = egg:swift#domain_remap
[filter:catch_errors]
use = egg:swift#catch_errors
[filter:cname_lookup]
use = egg:swift#cname_lookup
[filter:staticweb]
use = egg:swift#staticweb
[filter:tempurl]
use = egg:swift#tempurl
[filter:formpost]
use = egg:swift#formpost
[filter:name_check]
use = egg:swift#name_check
[filter:list-endpoints]
use = egg:swift#list_endpoints
[filter:proxy-logging]
use = egg:swift#proxy_logging
[filter:bulk]
use = egg:swift#bulk
[filter:slo]
use = egg:swift#slo
[filter:dlo]
use = egg:swift#dlo
[filter:container-quotas]
use = egg:swift#container_quotas
[filter:account-quotas]
use = egg:swift#account_quotas
[filter:gatekeeper]
use = egg:swift#gatekeeper
[filter:container_sync]
use = egg:swift#container_sync
[filter:xprofile]
use = egg:swift#xprofile
[filter:versioned_writes]
use = egg:swift#versioned_writes
[filter:copy]
use = egg:swift#copy
[filter:keymaster]
use = egg:swift#keymaster
encryption_root_secret = changeme
[filter:kms_keymaster]
use = egg:swift#kms_keymaster
[filter:encryption]
use = egg:swift#encryption
[filter:listing_formats]
use = egg:swift#listing_formats
[filter:symlink]
use = egg:swift#symlink

创建账户 Ring

[root@controller ~]# cd /etc/swift/
[root@controller ~]# swift-ring-builder account.builder create 18 1 1
[root@controller ~]# swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.200.20 --port 6202 --device sdc --weight 100
[root@controller ~]# swift-ring-builder account.builder
[root@controller ~]# swift-ring-builder account.builder rebalance

创建容器 Ring

[root@controller ~]# cd /etc/swift/
[root@controller ~]# swift-ring-builder container.builder create 10 1 1
[root@controller ~]# swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.200.20 --port 6201 --device sdc --weight 100
[root@controller ~]# swift-ring-builder container.builder
[root@controller ~]# swift-ring-builder container.builder rebalance

创建对象 Ring

[root@controller ~]# cd /etc/swift/
[root@controller ~]# swift-ring-builder object.builder create 10 1 1
[root@controller ~]# swift-ring-builder object.builder  add --region 1 --zone 1 --ip 192.168.200.20 --port 6200 --device sdc --weight 100 
[root@controller ~]# swift-ring-builder object.builder
[root@controller ~]# swift-ring-builder object.builder rebalance

Swift 存储系统的主配置文件

[root@controller ~]# vim /etc/swift/swift.conf
[swift-hash]
swift_hash_path_suffix = changeme
swift_hash_path_prefix = changeme
[storage-policy:0]
name = Policy-0
default = yes
aliases = yellow, orange
[swift-constraints]

修改文件属主

[root@controller ~]# chown -R root:swift /etc/swift

设置开机自启并重新启动

[root@controller ~]# systemctl enable openstack-swift-proxy.service memcached.service &&    systemctl restart openstack-swift-proxy.service memcached.service

安装和配置(compute)

准备

在安装和配置之前，请务必准备好块存储设备(sdc)

(1)安装软件包

[root@compute ~]# yum install xfsprogs rsync openstack-swift-account openstack-swift-container openstack-swift-object -y

(2)格式化/dev/sdc1

[root@compute ~]# mkfs.xfs -i size=1024 -f /dev/sdc
# -i size=1024 参数指定了 inode 的大小为 1024 字节
# inode 是文件系统中用于存储文件元数据的结构。

image-20230629194908798

(3)创建挂载点

[root@compute ~]# mkdir -p /swift/node/sdc

(4)编辑/etc/fstab文件，添加以下内容

[root@compute ~]# vim /etc/fstab
/dev/sdc /swift/node/sdc xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0

(5)挂载文件系统

[root@compute ~]# mount /dev/sdc /swift/node/sdc

(6)分发环配置文件

[root@compute ~]# scp controller:/etc/swift/*.ring.gz /etc/swift/

(7)创建或编辑/etc/rsyncd.conf文件以包含以下内容

[root@compute ~]# vim /etc/rsyncd.conf
pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
uid = swift
gid = swift
address = 127.0.0.1
[account]
path            = /swift/node
read only       = false
write only      = no
list            = yes
incoming chmod  = 0644
outgoing chmod  = 0644
max connections = 25
lock file =     /var/lock/account.lock
[container]
path            = /swift/node
read only       = false
write only      = no
list            = yes
incoming chmod  = 0644
outgoing chmod  = 0644
max connections = 25
lock file =     /var/lock/container.lock
[object]
path            = /swift/node
read only       = false
write only      = no
list            = yes
incoming chmod  = 0644
outgoing chmod  = 0644
max connections = 25
lock file =     /var/lock/object.lock
[swift_server]
path            = /etc/swift
read only       = true
write only      = no
list            = yes
incoming chmod  = 0644
outgoing chmod  = 0644
max connections = 5
lock file =     /var/lock/swift_server.lock

(8)启动rsyncd服务并将其配置为随系统启动而启动

[root@compute ~]# systemctl start rsyncd.service  && systemctl enable rsyncd.service

配置Swift对象存储服务组件

(1)编辑/etc/swift/account-server.conf文件

[DEFAULT]
bind_port = 6202
user = swift
swift_dir = /etc/swift
devices = /swift/node
mount_check = false
[pipeline:main]
pipeline = healthcheck recon account-server
[app:account-server]
use = egg:swift#account
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[account-replicator]
[account-auditor]
[account-reaper]
[filter:xprofile]
use = egg:swift#xprofile

(2)编辑/etc/swift/container-server.conf文件

[DEFAULT]
bind_port = 6201
user = swift
swift_dir = /etc/swift
devices = /swift/node
mount_check = false
[pipeline:main]
pipeline = healthcheck recon container-server
[app:container-server]
use = egg:swift#container
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
[container-replicator]
[container-updater]
[container-auditor]
[container-sync]
[filter:xprofile]
use = egg:swift#xprofile

(3)编辑/etc/swift/object-server.conf文件

[DEFAULT]
bind_port = 6200
user = swift
swift_dir = /etc/swift
devices = /swift/node
mount_check = false
[pipeline:main]
pipeline = healthcheck recon object-server
[app:object-server]
use = egg:swift#object
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock
[object-replicator]
[object-reconstructor]
[object-updater]
[object-auditor]
[filter:xprofile]
use = egg:swift#xprofile

(4)编辑/etc/swift/swift.conf文件

[swift-hash]
swift_hash_path_suffix = changeme
swift_hash_path_prefix = changeme
[storage-policy:0]
name = Policy-0
default = yes
aliases = yellow, orange
[swift-constraints]

(6)设置挂载点的属主

[root@compute ~]#  chown -R swift:swift /swift/node

(7)创建recon目录并设置属主

[root@compute ~]# mkdir -p /var/cache/swift
[root@compute ~]# chown -R root:swift /var/cache/swift
[root@compute ~]# chmod -R 775 /var/cache/swift
[root@compute ~]# chown -R root:swift /etc/swift

在存储节点上，启动Swift存储服务并设置为开机自启

[root@compute ~]# systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service openstack-swift-account-reaper.service openstack-swift-account-replicator.service
[root@compute ~]# systemctl restart openstack-swift-account.service openstack-swift-account-auditor.service openstack-swift-account-reaper.service openstack-swift-account-replicator.service

[root@compute ~]# systemctl enable openstack-swift-container.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-updater.service
[root@compute ~]# systemctl restart openstack-swift-container.service openstack-swift-container-auditor.service openstack-swift-container-replicator.service openstack-swift-container-updater.service

[root@compute ~]# systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service
[root@compute ~]# systemctl restart openstack-swift-object.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service

验证

(1)加载环境变量

[root@controller ~]# source admin-openrc.sh

(2)检查Swift服务状态

[root@controller ~]# swift stat
               Account: AUTH_0769b940829c4078a4aa573e83d6520c
            Containers: 0
               Objects: 0
                 Bytes: 0
       X-Put-Timestamp: 1688142619.69016
           X-Timestamp: 1688142619.69016
            X-Trans-Id: txb5117d325e1c49f3876da-00649f031a
          Content-Type: text/plain; charset=utf-8
X-Openstack-Request-Id: txb5117d325e1c49f3876da-00649f031a

(2)创建容器

使用命令创建容器 名称为 swift-test

[root@controller ~]# openstack container create swift-test
+---------------------------------------+------------+------------------------------------+
| account                               | container  | x-trans-id                         |
+---------------------------------------+------------+------------------------------------+
| AUTH_0769b940829c4078a4aa573e83d6520c | swift-test | txe78d3ff78bb1422ba1bbf-00649f03cd |
+---------------------------------------+------------+------------------------------------

(3)查看容器

[root@controller ~]#  openstack container list
+------------+
| Name       |
+------------+
| swift-test |
+------------+

(4)创建对象

[root@controller ~]# mkdir test
[root@controller ~]# echo "I am csq,  welcome Swift\!" >>  test/csq.txt
[root@controller ~]#  openstack object create swift-test  test/csq.txt
+--------------+------------+----------------------------------+
| object       | container  | etag                             |
+--------------+------------+----------------------------------+
| test/csq.txt | swift-test | 83da8bfa7f0e1d023c546e105a35cd88 |
+--------------+------------+----------------------------------+

(5)查看对象

[root@controller ~]# openstack object list swift-test
+--------------+
| Name         |
+--------------+
| test/csq.txt |
+--------------+

(6)下载对象

[root@controller ~]# rm -rf test/
[root@controller ~]# openstack object save swift-test test/csq.txt
[root@controller ~]# ls test/
csq.txt

分片存储

(1)创建容器

[root@controller ~]# openstack container create swift-test2
+---------------------------------------+-------------+------------------------------------+
| account                               | container   | x-trans-id                         |
+---------------------------------------+-------------+------------------------------------+
| AUTH_0769b940829c4078a4aa573e83d6520c | swift-test2 | tx5c548a18c29646ccadde1-00649f05d8 |
+---------------------------------------+-------------+------------------------------------+

(2)上传镜像并分片存储

每个片段大小为10M

[root@controller ~]# swift upload swift-test2 -S 10000000 image/cirros-0.3.4-x86_64-disk.img 
image/cirros-0.3.4-x86_64-disk.img segment 1
image/cirros-0.3.4-x86_64-disk.img segment 0
image/cirros-0.3.4-x86_64-disk.img

查看cirros镜像的存储路径

[root@controller ~]#  swift stat swift-test2 image/cirros-0.3.4-x86_64-disk.img 
               Account: AUTH_0769b940829c4078a4aa573e83d6520c
             Container: swift-test2
                Object: image/cirros-0.3.4-x86_64-disk.img
          Content Type: application/octet-stream
        Content Length: 13287936
         Last Modified: Fri, 30 Jun 2023 16:43:29 GMT
                  ETag: "cca17a689bfcd70118fabc833af0d033"
              Manifest: swift-test2_segments/image/cirros-0.3.4-x86_64-disk.img/1687999427.910558/13287936/10000000/
            Meta Mtime: 1687999427.910558
         Accept-Ranges: bytes
           X-Timestamp: 1688143408.18578
            X-Trans-Id: tx0b6fbefac2c24e69a998e-00649f0681
X-Openstack-Request-Id: tx0b6fbefac2c24e69a998e-00649f0681

查看存储路径中的数据片

[root@controller ~]# swift list swift-test2_segments
image/cirros-0.3.4-x86_64-disk.img/1687999427.910558/13287936/10000000/00000000
image/cirros-0.3.4-x86_64-disk.img/1687999427.910558/13287936/10000000/00000001

可以看到的是镜像上传到Swift存储中被分片存储了，单个存储片的大小为10M

猜你喜欢

转载自blog.csdn.net/qq_52089863/article/details/131485777
今日推荐
开源日报 | Chrome内置Gemini的意义不在于Gemini；中国AI追随之路的五大误区；ECharts创始人“下海”养鱼；谷歌I/O开发者大会什么都有，只是没有惊喜
微软回应中国区AI团队“打包赴美”传闻
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
周排行
laravle中orm简单的增删改查
文本分类 特征选取之CHI开方检验
Spark核心编程-WordCount
大数据开发实战系列之电信客服(1)
读书笔记 - 把时间当作朋友 by 李笑来
python 笔记--if else
SpringBoot/Mybatis/Druid, 多数据源MultiDataSource配置思路
排序三个整数
redis集群搭建【2】-Windows中Redis集群搭建
STM32F030驱动TM1650点亮4联数码管
每日归档
更多
2024-05-16(6)
2024-05-15(24)
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022