Hello everyone , and welcome to this blog. The blogger is a novice who has just entered the big data industry. He uses his free time to share what he has learned and helps students who are just like the blogger who is just in the beginning stage. The level is not high. , If there are any mistakes and omissions, please feel free to enlighten me. At present, the personal blog only has CSDN:,
https://zhenyu.blog.csdn.net/
thank you for your support, thank you, and
send you a sentence: today’s matter, todayThis blog mainly explains: Data synchronization tool: Canal installation
aims:
- Install Canal
Important version update instructions
------ 1.Canal 1.1.x version ( release_note ), there is a big breakthrough in performance and function. Important improvements include:
- The overall performance test & optimization has improved by 150%. #726 Reference: Performance
- Native support for prometheus monitoring #765 Prometheus QuickStart
- Native support for kafka message delivery #695 Canal Kafka/RocketMQ QuickStart
- Native support for binlog subscription of aliyun rds (solve automatic active/standby switching/oss binlog offline analysis) Reference: Aliyun RDS QuickStart
- Native support for docker image#801 Reference: Docker QuickStart
2.canal 1.1.4 version, ushered in the most important WebUI capabilities, introduced the canal-admin project, supports WebUI-oriented canal dynamic management capabilities, and supports online white-screen operation and maintenance capabilities such as configuration, tasks, logs, etc. Specific document: Canal Admin Guide
Note: The version used in this study canal1.0.24
Environmental requirements
- Download: click to download Cancal_ extraction code: yujv
- Install ZooKeeper
- unzip
mkdir /export/servers/canal
tar -zxvf canal.deployer-1.0.24.tar.gz -C /export/servers/canal/
- After decompression is complete, enter the /export/servers/canal/ directory, you can see the following structure
[root@node1 canal]# tree conf/
conf/
├── canal.properties
├── example
│ └── instance.properties
├── logback.xml
└── spring
├── default-instance.xml
├── file-instance.xml
├── group-instance.xml
├── local-instance.xml
└── memory-instance.xml
- First look
canal.properties
at the first four configuration items of the common attribute:
canal.id= 1
canal.ip=
canal.port= 11111
canal.zkServers=
canal.id is the number of canal. In a cluster environment, the id of different canal is different. Note that it is different from the server_id of MySQL.
Ip is not specified here, and the default is this machine, for example, the above is 192.168.1.120, and the port number is 11111. zk is used for canal cluster.
- Let's look at the configuration related
canal.properties
to destinations :
#################################################
######### destinations #############
#################################################
canal.destinations = example
canal.conf.dir = ../conf
canal.auto.scan = true
canal.auto.scan.interval = 5
canal.instance.global.mode = spring
canal.instance.global.lazy = false
canal.instance.global.spring.xml = classpath:spring/file-instance.xml
------ Here canal.destinations
= example
can set multiple, such as example1, example2, you need to create two corresponding folders, and each folder has a instance.properties
file.
- The global canal instance management uses spring, here will
file-instance.xml
eventually instantiate all the destinations instances:
<!-- properties -->
<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
<property name="ignoreResourceNotFound" value="true" />
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
<property name="locationNames">
<list>
<value>classpath:canal.properties</value> <value>classpath:${canal.instance.destination:}/instance.properties</value>
</list>
</property>
</bean>
<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer">
<property name="propertyEditorRegistrars">
<list>
<ref bean="socketAddressEditor" />
</list>
</property>
</bean>
<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
<property name="destination" value="${canal.instance.destination}" />
<property name="eventParser">
<ref local="eventParser" />
</property>
<property name="eventSink">
<ref local="eventSink" />
</property>
<property name="eventStore">
<ref local="eventStore" />
</property>
<property name="metaManager">
<ref local="metaManager" />
</property>
<property name="alarmHandler">
<ref local="alarmHandler" />
</property>
</bean>
------For example, if canal.instance.destination
equal to example, the example/instance.properties
configuration file will be loaded
- Modify the instance configuration file
vi conf/example/instance.properties
## mysql serverId,这里的slaveId不能和myql集群中已有的server_id一样
canal.instance.mysql.slaveId = 1234
# 按需修改成自己的数据库信息
#################################################
...
canal.instance.master.address=192.168.1.120:3306
# username/password,数据库的用户名和密码
...
canal.instance.dbUsername = root
canal.instance.dbPassword = 123456
#################################################
- start up
sh bin/startup.sh
- View server log
vi logs/canal/canal.log
---------------------------------
2013-02-05 22:45:27.967 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher - ## start the canal server.
2013-02-05 22:45:28.113 [main] INFO com.alibaba.otter.canal.deployer.CanalController - ## start the canal server[10.1.29.120:11111]
2013-02-05 22:45:28.210 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher - ## the canal server is running now ......
- View instance log
vi logs/example/example.log
-----------------------------------------
2013-02-05 22:50:45.636 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [canal.properties]
2013-02-05 22:50:45.641 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [example/instance.properties]
2013-02-05 22:50:45.803 [main] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start CannalInstance for 1-example
2013-02-05 22:50:45.810 [main] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start successful....
- shut down
sh bin/stop.sh
summary
--------The above are the installation steps of Canal.
Thank you for your support. If there is anything wrong, please give us feedback in time. Remember to like and support!