MongoDB Sharding Cluster是一种可以水平扩展的模式,在数据量很大时特给力,一直想研究一下,要构建一个MongoDB Sharding Cluster,需要三种角色:
1. Shard Server: mongod实例,用于存储实际的数据块,实际生产环境中一个Shard Server角色可由几台机器组个一个Replica Set承担,防止主机单点故障。
2. Config Server: mongod实例,存储了整个Cluster Metadata,其中包括Chunk信息。
3. Route Server: mongos实例,前端路由,客户端由此接入,且让整个集群看上去像单一数据库,前端应用可以透明使用。
Sharding架构图如下:
因为是实验环境,我这里只是用一台Linux服务器,基本信息如下:
1. 3个分片Sharding
2. 每一个分片由3个节点构成1主2备的Replica Set
3. 3个配置节点Configsvr
4. 1个路由节点Mongos
分片复制集A(三个分片节点构成一个复制集):127.0.0.1:10000 127.0.0.1:10001 127.0.0.1:10002
分片复制集B(三个分片节点构成一个复制集):127.0.0.1:20000 127.0.0.1:20001 127.0.0.1:20002
分片复制集C(三个分片节点构成一个复制集):127.0.0.1:30000 127.0.0.1:30001 127.0.0.1:30002
Configsvr(三个配置服务器节点):127.0.0.1:40000 127.0.0.1:40001 127.0.0.1:40002
Mongos(一个路由节点):127.0.0.1:50000
详细操作步骤如下:
1. 下载并安装MongoDB,并创建MongoDB用户
- tar zxvf mongodb-linux-i686-2.0.4.tgz -C /usr/local/
- mv /usr/local/mongodb-linux-i686-2.0.4/ /usr/local/mongodb/
- ln -s /usr/local/mongodb/bin/* /usr/bin/
-
- groupadd -g 20001 mongodb
- useradd -u 20001 -g mongodb mongodb
-
- su - mongodb
2. 创建相关目录
- mkdir -p /home/mongodb/data/shard1/r0
- mkdir -p /home/mongodb/data/shard1/r1
- mkdir -p /home/mongodb/data/shard1/r2
- mkdir -p /home/mongodb/data/shard1/config
-
- mkdir -p /home/mongodb/data/shard2/r0
- mkdir -p /home/mongodb/data/shard2/r1
- mkdir -p /home/mongodb/data/shard2/r2
- mkdir -p /home/mongodb/data/shard2/config
-
- mkdir -p /home/mongodb/data/shard3/r0
- mkdir -p /home/mongodb/data/shard3/r1
- mkdir -p /home/mongodb/data/shard3/r2
- mkdir -p /home/mongodb/data/shard3/config
3. 创建Sharding和Replica Set
(1) 创建并配置第一组
- mongod -shardsvr -replSet shard1 -port 10000 -dbpath /home/mongodb/data/shard1/r0 -oplogSize 64 -logpath /home/mongodb/data/shard1/r0.log -logappend -fork
- mongod -shardsvr -replSet shard1 -port 10001 -dbpath /home/mongodb/data/shard1/r1 -oplogSize 64 -logpath /home/mongodb/data/shard1/r1.log -logappend -fork
- mongod -shardsvr -replSet shard1 -port 10002 -dbpath /home/mongodb/data/shard1/r2 -oplogSize 64 -logpath /home/mongodb/data/shard1/r2.log -logappend -fork
-
- mongo 127.0.0.1:10000/admin
-
- config={_id: 'shard1', members: [{_id: 0, host: '127.0.0.1:10000'}, {_id: 1, host: '127.0.0.1:10001'}, {_id: 2, host: '127.0.0.1:10002'}]}
- rs.initiate(config);
(2) 创建并配置第二组
- mongod -shardsvr -replSet shard2 -port 20000 -dbpath /home/mongodb/data/shard2/r0 -oplogSize 64 -logpath /home/mongodb/data/shard2/r0.log -logappend -fork
- mongod -shardsvr -replSet shard2 -port 20001 -dbpath /home/mongodb/data/shard2/r1 -oplogSize 64 -logpath /home/mongodb/data/shard2/r1.log -logappend -fork
- mongod -shardsvr -replSet shard2 -port 20002 -dbpath /home/mongodb/data/shard2/r2 -oplogSize 64 -logpath /home/mongodb/data/shard2/r2.log -logappend -fork
-
- mongo 127.0.0.1:20000/admin
-
- config={_id: 'shard2', members: [{_id: 0, host: '127.0.0.1:20000'}, {_id: 1, host: '127.0.0.1:20001'}, {_id: 2, host: '127.0.0.1:20002'}]}
- rs.initiate(config);
(3) 创建并配置第三组
- mongod -shardsvr -replSet shard3 -port 30000 -dbpath /home/mongodb/data/shard3/r0 -oplogSize 64 -logpath /home/mongodb/data/shard3/r0.log -logappend -fork
- mongod -shardsvr -replSet shard3 -port 30001 -dbpath /home/mongodb/data/shard3/r1 -oplogSize 64 -logpath /home/mongodb/data/shard3/r1.log -logappend -fork
- mongod -shardsvr -replSet shard3 -port 30002 -dbpath /home/mongodb/data/shard3/r2 -oplogSize 64 -logpath /home/mongodb/data/shard3/r2.log -logappend -fork
-
- mongo 127.0.0.1:30000/admin
-
- config={_id: 'shard3', members: [{_id: 0, host: '127.0.0.1:30000'}, {_id: 1, host: '127.0.0.1:30001'}, {_id: 2, host: '127.0.0.1:30002'}]}
- rs.initiate(config);
4. 启动三个配置服务节点Configsvr
- mongod -configsvr -dbpath /home/mongodb/data/shard1/config -port 40000 -logpath /home/mongodb/data/shard1/config.log -logappend -fork
- mongod -configsvr -dbpath /home/mongodb/data/shard2/config -port 40001 -logpath /home/mongodb/data/shard2/config.log -logappend -fork
- mongod -configsvr -dbpath /home/mongodb/data/shard3/config -port 40002 -logpath /home/mongodb/data/shard3/config.log -logappend -fork
5. 启动一个路由点Mongos
- mongos -configdb 127.0.0.1:40000,127.0.0.1:40001,127.0.0.1:40002 -port 50000 -chunkSize 5 -logpath /home/mongodb/data/mongos.log -logappend -fork
6. 配置分片
- mongo 127.0.0.1:50000/admin
-
- db.runCommand({addshard:"shard1/127.0.0.1:10000,127.0.0.1:10001,127.0.0.1:10002",name:"ShardSetA"})
- db.runCommand({addshard:"shard2/127.0.0.1:20000,127.0.0.1:20001,127.0.0.1:20002",name:"ShardSetB"})
- db.runCommand({addshard:"shard3/127.0.0.1:30000,127.0.0.1:30001,127.0.0.1:30002",name:"ShardSetC"})
7. 查看配置结果
- mongos> db.runCommand({listshards: 1})
- {
- "shards" : [
- {
- "_id" : "ShardSetA",
- "host" : "shard1/127.0.0.1:10000,127.0.0.1:10001,127.0.0.1:10002"
- },
- {
- "_id" : "ShardSetB",
- "host" : "shard2/127.0.0.1:20000,127.0.0.1:20001,127.0.0.1:20002"
- },
- {
- "_id" : "ShardSetC",
- "host" : "shard3/127.0.0.1:30000,127.0.0.1:30001,127.0.0.1:30002"
- }
- ],
- "ok" : 1
- }
- mongos>
8. 如果不激活数据库分片,数据库只会存在一个Shard中,一旦激活数据库分片,数据库中不同的Collection将被存放在不同的Shard上,但一个Collection仍旧存放在同一个Shard上,要使单个Collection也分片,还需单独对Collection作些操作。
- mongos> db.runCommand({enablesharding:"test"});
- { "ok" : 1 }
9. 要使单个Collection也分片存储,需要给Collection指定一个分片key,通过以下命令操作:
- mongos> db.runCommand({shardcollection:"test.c1",key:{id:1}})
- { "collectionsharded" : "test.c1", "ok" : 1 }
10. 查看Collection分片信息
- mongos> for (var i = 1; i <= 13210; i++) db.c1.save({id:i,value1:"1234567890",value2:"1234567890",value3:"1234567890",value4:"1234567890"});
- mongos> db.c1.stats();
- {
- "sharded" : false,
- "primary" : "config",
- "ns" : "admin.c1",
- "count" : 13210,
- "size" : 1690880,
- "avgObjSize" : 128,
- "storageSize" : 2793472,
- "numExtents" : 5,
- "nindexes" : 1,
- "lastExtentSize" : 2097152,
- "paddingFactor" : 1,
- "flags" : 1,
- "totalIndexSize" : 457856,
- "indexSizes" : {
- "_id_" : 457856
- },
- "ok" : 1
- }