背景:目前有两台服务器hadoop100 和 hadoop101,需要将hadoop101中 /opt/module/data/test 目录下的数据实时同步到 hadoop100 的/backup/目录下,通过rsync+inotify的形式实现
rsync的配置参照上一篇:https://blog.csdn.net/weixin_43190860/article/details/91043866
一、inotify简介
Inotify 是一个 Linux 内核特性,它监控文件系统,并且及时向专门的应用程序发出相关的事件警告,比如删除、读、写和卸载操作等。您还可以跟踪活动的源头和目标等细节。
二、inotify安装
①检测服务器是否支持inotify机制
uname -r ##内核版本,从kernel2.6.13开始支持
ls -l /proc/sys/fs/inotify/ ##查看是否支持
-rw-r–r-- 1 root root 0 6月 10 10:31 max_queued_events
-rw-r–r-- 1 root root 0 6月 10 10:31 max_user_instances
-rw-r–r-- 1 root root 0 6月 10 10:31 max_user_watches
显示这三个文件表示支持
②检查是否有安装inotify
rpm -qa inotify-tools
没有就先安装epol源
yum.repos.d]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
安装inotify-tools
yum install inotify-tools -y
备注:如何查看通过yum安装的包在哪儿:rpm -ql 文件包名
③参数说明
扫描二维码关注公众号,回复:
6409935 查看本文章
Options:
-h|--help Show this help text.
@<file> Exclude the specified file from being watched.
--exclude <pattern>
Exclude all events on files matching the
extended regular expression <pattern>.
--excludei <pattern>
Like --exclude but case insensitive.
-m|--monitor Keep listening for events forever. Without
this option, inotifywait will exit after one
event is received.
-d|--daemon Same as --monitor, except run in the background
logging events to a file specified by --outfile.
Implies --syslog.
-r|--recursive Watch directories recursively.
--fromfile <file>
Read files to watch from <file> or `-' for stdin.
-o|--outfile <file>
Print events to <file> rather than stdout.
-s|--syslog Send errors to syslog rather than stderr.
-q|--quiet Print less (only print events).
-qq Print nothing (not even events).
--format <fmt> Print using a specified printf-like format
string; read the man page for more details.
--timefmt <fmt> strftime-compatible format string for use with
%T in --format string.
-c|--csv Print events in CSV format.
-t|--timeout <seconds>
When listening for a single event, time out after
waiting for an event for <seconds> seconds.
If <seconds> is 0, inotifywait will never time out.
-e|--event <event1> [ -e|--event <event2> ... ]
Listen for specific event(s). If omitted, all events are
listened for.
Exit status:
0 - An event you asked to watch for was received.
1 - An event you did not ask to watch for was received
(usually delete_self or unmount), or some error occurred.
2 - The --timeout option was given and no events occurred
in the specified interval of time.
Events:
access file or directory contents were read
modify file or directory contents were written
attrib file or directory attributes changed
close_write file or directory closed, after being opened in
writeable mode
close_nowrite file or directory closed, after being opened in
read-only mode
close file or directory closed, regardless of read/write mode
open file or directory opened
moved_to file or directory moved to watched directory
moved_from file or directory moved from watched directory
move file or directory moved to or from watched directory
create file or directory created within watched directory
delete file or directory deleted within watched directory
delete_self file or directory was deleted
unmount file system containing file or directory unmounted
常用参数:
① -r|--recursive Watch directories recursively. ##递归查询目录
② -q|--quiet Print less (only print events) ##打印很少的信息,仅仅打印监控相关的信息
③-m|--monitor Keep listening for events forever. Without
this option, inotifywait will exit after one
event is received. ##始终保持事件监听状态
④--excludei <pattern>
Like --exclude but case insensitive. ##排除文件或目录时,不区分大小写
⑤--timefmt <fmt> strftime-compatible format string for use with
%T in --format string. ##指定时间的输出格式
④使用inotifywait监控指定目录
命令:
/usr/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e modify,delete,create,close_write /opt/module/data/test
编写脚本,如果inotify监控到文件的变化,使用rsync进行同步
#!/bin/bash
inotify=/usr/bin/inotifywait
$inotify -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e modify,delete,create,close_write /opt/module/data/test | while read file
do
rsync -avz /opt/module/data/test diao@hadoop100::backup --password-file=/etc/rsyncd.password
done