Use mongo-connector to synchronize data from mongo to elasticsearch step (windows implementation)

From the needs of the project, the data of mongo is synchronized to elasticsearch (hereinafter referred to as ES). Let ES do the retrieval. Finally realized through research. The specific steps are as follows:

 

A.mongo replica set configuration

 This article configures the use of database version: 3.0.5 64-bit version

1. Copy the three-point mongodb (bin folder and its subdirectories) to the server. The directories are as follows:

D:\mongo_db\mongo_master /*Mongo master library in the current directory: localhost:27311*/

D:\mongo_db\mongo_slave /*Mongo in the current directory as a slave library: localhost:27321*/

D:\mongo_db\mongo_arbiter /*Mongo in the current directory as the arbiter server: localhost:27331*/

2. The master/slave/arbitration creation file directory is as follows:

D:\mongo_db\mongo_master\data\db

D:\mongo_db\mongo_master\data\log

 

D:\mongo_db\mongo_slave\data\db

D:\mongo_db\mongo_slave\data\log

 

D:\mongo_db\mongo_arbiter\data\db

D:\mongo_db\mongo_arbiter\data\log

 

3. Create the main database database service:

cd D:\mongo_db\mongo_master\bin

Excuting an order:

mongod --replSet shard1 --port 27311 --logpath "D:\mongo_db\mongo_master\data\log\MongoDB.log" --logappend --dbpath "D:\mongo_db\mongo_master\data\db" --serviceName "mongo_master" --serviceDisplayName "mongo_master" --install

4. Create a slave database service:

cd D:\mongo_db\mongo_slave\bin

Excuting an order:

mongod --replSet shard1 --port 27321 --logpath "D:\mongo_db\mongo_slave\data\log\MongoDB.log" --logappend --dbpath "D:\mongo_db\mongo_slave\data\db" --serviceName "mongo_slaver" --serviceDisplayName "mongo_slaver" --install

5. Create the arbitration library service:

cd D: \ mongo_db \ ​​mongo_arbiter \ bin

mongod --replSet shard1 --port 27331 --logpath "D:\mongo_db\mongo_arbiter\data\log\MongoDB.log" --logappend --dbpath "D:\mongo_db\mongo_arbiter\data\db" --serviceName "mongo_arbiter" --serviceDisplayName "mongo_arbiter" --install

 

6. Start Master/Slave/Arbitration

Enter in the cmd console:

net start mongo_master

do not start mongo_slaver

net start mongo_arbiter

 

7. Log in to the main library:

cd D:\mongo_db\mongo_slave\bin

mongo localhost:27311

>use admin

>cfg={_id:"shard1",members:[{_id:0,host:"localhost:27311",priority:1},{_id:1,host:"localhost:27321",priority:1},{_id:2,host:"localhost:27331",arbiterOnly:true}]}

>rs.initiate(cfg)

>rs.status()//This command verifies that the replica set is added successfully

as follows:

 

 

 At this point, the mongo replica set configuration is complete.

 

B. Install python

Install python-3.5.3-amd64.exe 

Install mongo-connector command: pip install elastic-doc-manager[elastic2]

 

C. Installation: ElasticSearch

D:\software\elasticsearch-2.4.0

And start ElasticSearch: Double click: D:\software\elasticsearch-2.4.0\bin\elasticsearch.bat

 

E. Start mongo-connector:

mongo-connector --auto-commit-interval=0 -m localhost:27311 -t localhost:9200 -d elastic2_doc_manager

 

F: Log in to the main library to create a piece of data:

>use person

>db.person.insert({"name":"jack","age":30})

>db.person.find() //Check if the save is successful

 

G: Enter in the browser address bar: http://localhost:9200/_cat/indices?v You can see that there is already a person under the index column. It proves that it has been synchronized to ES.

H: Log in to the mongo slave library:

D:\mongo_db\mongo_slave\bin>mongo localhost:27321 MongoDB shell version: 3.0.5 connecting to: localhost:27321/test shard1:SECONDARY> use person switched to db person shard1:SECONDARY> db.person.find() Error: error: { "$err" : "not master and slaveOk=false", "code" : 13435 } shard1:SECONDARY> rs.slaveOk(); shard1:SECONDARY> db.person.find() { "_id" : ObjectId("598417b9017cd7642360634d"), "name" : "Johon", "age" : "30" }

 

I: The whole process is over.

 

 

 

 

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=326182079&siteId=291194637