Practice of the whole network scheduled backup crond + rsync (timing)
1. Verify that success to make a mark touch flag 2.md5sum do the whole network identification
#######脚本开始##############
cd /
NAME=`hostname`
IP=`ifconfig eth1 | awk -F " " 'NR==2{print $2}'` 取IP
tar cvzhf $IP_$NAME_file test1 test2 test3 &&\
rsync avz $IP_$NAME_file [email protected]::backup &&\
md5sum file >> $IP_$NAME_flag`date -I` 创建FLAG文件内容为md5sum &&\
rsync avz $IP_$NAME_flag`date -I` [email protected]::backup
######脚本结束##############Server available $ IP_ $ NAME_flag`date -I` do verify the file name
Network-wide real-time backup practices
need rsyncd daemon architecture,
when found in a local file changes occur, rsync pushed to the backup server
inotify (this software a lot of) the principle of monitoring disk change sersync lrsyncd
inotify-tools asynchronous file system monitor
/ proc / sys / fs / inotify three profile
adjusted according to actual situation following three documents
max_queued_events the maximum number of events in the queue
the maximum number of each user entity max_user_instances run command number
max_user_watches the maximum number of monitoring
yum install -y inotify
包含下面两个命令
inotifywait
inotifywatch
inotifywait -mrq --timefmt '%d/%m/%y/%H:%M' --format '%T%w%f' -e modify,create,delete /nfsdata``
-m monitor 不加 只监视一次
-r 递归 不加只监视当前一层目录
-q quiet模式,不加输出信息多
--timefmt '%y %m %d %H %M' 指定时间格式 '' "" 目前测试效果一样
年 月 日 时 分
--format '%T%w%f' 输出监控结果的格式 stand output/stand error
%T 代表使用--timefmt格式,不加则没有日期
%w 显示目录 不加为event号
%f 显示文件名 不加为空
-e 指定监控的动作 create delete modify attrib
close_write(改) open moved_to moved_from
move umount 等 具体man
/nfsdata /var/log/lastlog 监控的目录或文件
inotifywait 触发rsync推送
#############################脚本开始
#!/bin/bash
Path=/nfsdata
Ip=10.3.3.3
inotifywait -mr --timefmt '%Y/%m/%d %H:%M ' --format '%T%w%f' -e modify,close_write $Path/ \
|while read line
###这里使用重定向，将文件内容输入到while命令，while命令每次使用read从输入中读取一行数据。 用 line file 都可以
do
cd /nfsdata
rsync -avz $Path/ --delete rsync_backup@$10.3.3.3::backup --password-file=/etc/rsync.pp
done
################################脚本结束inotify advantages: file system event monitoring changes in real-time data synchronization tool
Cons: Concurrent greater than 200 (100K), plus single-threaded concurrency will be delayed & concurrency.
Multi-process with sersync
High concurrent data synchronization scheme
1.inotify rsync file level inefficient
2.drbd based file system level block disadvantages: Preparation of node data unavailable
Third-party software synchronization MongoDB Oracle MySQL
4. Write program double, two servers directly written
5. business logic solving separate read and write
6.NFS cluster (double write main memory, storing backup inotify (sersync) + rsync master device is not solved to find the front-end server apparatus delays do inotify (sersync) + rsync