ELK环境准备
配置：2核心2G内存
规划如下：
ip地址：
192.168.59.130：
jdk
kibana
elasticsearch
192.168.59.131:
jdk
logstash
1.关闭防火墙
systemctl stop firewalld
setenforce 0
2.时间同步
yum -y install ntpdate
ntpdate pool.ntp.org
ELK相关包链接：https://pan.baidu.com/s/112s9cetAG0PuCjENr0hT4Q
提取码：avsz
3.两台安装JDK
unzip ELK.zip
yum -y install jdk-8u131-linux-x64_.rpm
java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
4：安装elasticsearch
192.168.59.130 安装elasticsearch
yum -y install elasticsearch-6.6.2.rpm
vim /etc/elasticsearch/elasticsearch.yml
运行elasticsearch服务
systemctl enable elasticsearch
systemctl start elasticsearch
验证是否运行
tailf /var/log/elasticsearch/fncxy.log
显示info成功
5：安装logstash
192.168.59.131下安装logstash
yum -y install logstash-6.6.0.rpm
vim /etc/logstash/conf.d/msg.conf
配置文件
开启服务
systemctl start logstash
稍等一分钟查看
ss -nltp | grep 9600
LISTEN 0 50 [::ffff:127.0.0.1]:9600 [::]:* users:(("java",pid=14233,fd=88))
chmod 777 /var/log -R
6 192.168.59.130安装kibana
yum -y install kibana-6.6.2-x86_64.rpm
vim /etc/kibana/kibana.yml
systemctl start kibana
ss -nltp |grep 5601
LISTEN 0 128 192.168.59.130:5601 *:* users:(("node",pid=14381,fd=18))
7：收集nginx日志
yum -y install epel-release
yum -y install nginx
yum -y install httpd-tools
ab -n 1000 -c 1000 http://192.168.1.8/index.html
-n:请求数
-c:并发数
注意：并发数不能大于请求数！！
input{
file{
path => "/var/log/nginx/access.log"
type => 'nginx-log'
start_position => "beginning"
}
}
filter {
grok{
match => {
"message" => "%{NGX}"}
}
}
output{
elasticsearch{
hosts => "192.168.10.130:9200"
index => "nginx_log-%{+YYYY.MM.dd}"
}
}
vim /etc/logstash/pipelines.yml
- pipeline.id: nginx
path.config: "/etc/logstash/conf.d/nginx.conf"
vim /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/nginx
定义正则
NGX %{
IPORHOST:client_ip} (%{
USER:ident}|- ) (%{
USER:auth}|-) \[%{
HTTPDATE:timestamp}\] "(?:%{WORD:verb} (%{NOTSPACE:request}|-)(?: HTTP/%{NUMBER:http_version})?|-)" %{
NUMBER:status} (?:%{
NUMBER:bytes}|-) "(?:%{URI:referrer}|-)" "%{GREEDYDATA:agent}"
chmod 777 /var/log -R
重启logstash
systemctl restart logstash
状态码
pv值
前10访问
访问趋势图
8 邮件报警
Python3.6及elasticalert链接：https://pan.baidu.com/s/18IBQejJE7rKr9y9pHMyeiA
提取码：y8iu
1.安装Python编译环境
yum -y install gcc gcc-c++ openssl openssl-devel
tar zxvf Python-3.6.2.tgz
cd Python-3.6.2
./configure --prefix=/usr/local/python3 --with-openssl
make && make install
2.设置软链接
rm -rf /usr/bin/python #删除以前版本的py
ln -s /usr/local/python3/bin/python3.6 /usr/bin/python #新的3.6上来顶替
ln -s /usr/local/python3/bin/pip3.6 /usr/bin/pip #新的上来顶替
3.修复yum
vim /usr/bin/yum 将python 修改为python2
vim /usr/libexec/urlgrabber-ext-down 将python 修改为python2
3 安装alert 插件
tar zxvf v0.2.1_elasticalert.tar.gz
mv elastalert-0.2.1/ /usr/local/elastalert
安装依赖包：
cd /usr/local/elastalert
pip install "elasticsearch<7,>6"
pip install -r requirements.txt -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
python setup.py install
会生成以下命令
-rwxr-xr-x. 1 root root 422 8月 19 03:13 elastalert-create-index
-rwxr-xr-x. 1 root root 396 8月 19 03:13 elastalert
-rwxr-xr-x. 1 root root 416 8月 19 03:13 elastalert-test-rule
-rwxr-xr-x. 1 root root 430 8月 19 03:13 elastalert-rule-from-kibana
ln -s /usr/local/python3/bin/elastalert* /usr/bin/
cd /usr/local/elastalert
改名字：
mv config.yaml.example config.yaml
vim config.yaml
cd /usr/local/elastalert/example_rules #进入模板库
mv example_frequency.yaml nginx_frequency.yaml #为自己的监控模板改个名字
vim nginx_frequency.yaml #修改模板配置文件
配置邮件相关的配置文件
vim email_auth.yaml
配置Linux自带邮件
yum -y install mailx
vim /etc/mail.rc
开启elastalert服务
elastalert --config /usr/local/elastalert/config.yaml --rule /usr/local/elastalert/example_rules/nginx_frequency.yaml --verbose
