一步步建立 ELK log分析平台 --- elasticsearch 6.5.4 安装

[root@iZj6c8tmr3yb0ntvym98ryZ elasticsearch]# lssysctl -p
-bash: lssysctl: command not found
[root@iZj6c8tmr3yb0ntvym98ryZ elasticsearch]# sysctl -p
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
vm.swappiness = 0
net.ipv4.neigh.default.gc_stale_time = 120
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_synack_retries = 2
kernel.sysrq = 1
vm.max_map_count = 262144

4.修改文件/etc/security/limits.conf

注意，在阿里云的机器上默认用的值是 65535，注意修改到 65536

vim /etc/security/limits.conf

* hard nofile 65536
* soft nofile 65536
 
* soft nproc 2048
* hard nproc 4096

5. 修改 /etc/security/limits.d/20-nproc.conf

# vi /etc/security/limits.d/20-nproc.conf

# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     4096
root       soft    nproc     unlimited

elasticsearch 5.x之后版本不允许root用户启动，所以我们需要添加一个es用户

6. 添加 es 用户

#groupadd elasticsearch
#useradd es
#usermod -G elasticsearch es
#chown -R es elasticsearch-6.5.4/
#chgrp -R elasticsearch  elasticsearch-6.5.4/

7. 启动ES

整个安装过程到这里为止已经全部结束，切换到es用户，启动elasticsearch

[es@iZj6c8tmr3yb0ntvym98ryZ bin]$ ./elasticsearch
[2019-01-19T15:38:35,769][INFO ][o.e.e.NodeEnvironment    ] [node-HK143] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [86.9gb], net total_space [98.3gb], types [rootfs]
[2019-01-19T15:38:35,776][INFO ][o.e.e.NodeEnvironment    ] [node-HK143] heap size [1007.3mb], compressed ordinary object pointers [true]
[2019-01-19T15:38:35,777][INFO ][o.e.n.Node               ] [node-HK143] node name [node-HK143], node ID [0-iZEA3xQwiCW5nDjvkhcQ]
[2019-01-19T15:38:35,778][INFO ][o.e.n.Node               ] [node-HK143] version[6.5.4], pid[17769], build[default/tar/d2ef93d/2018-12-17T21:17:40.758843Z], OS[Linux/3.10.0-693.2.2.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_191/25.191-b12]
[2019-01-19T15:38:35,778][INFO ][o.e.n.Node               ] [node-HK143] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.JUEFmWF7, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/local/apps/elasticsearch, -Des.path.conf=/usr/local/apps/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [aggs-matrix-stats]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [analysis-common]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [ingest-common]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [lang-expression]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [lang-mustache]
[2019-01-19T15:38:38,490][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [lang-painless]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [mapper-extras]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [parent-join]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [percolator]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [rank-eval]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [reindex]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [repository-url]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [transport-netty4]
[2019-01-19T15:38:38,491][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [tribe]
[2019-01-19T15:38:38,492][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-ccr]
[2019-01-19T15:38:38,492][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-core]
[2019-01-19T15:38:38,492][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-deprecation]
[2019-01-19T15:38:38,492][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-graph]
[2019-01-19T15:38:38,492][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-logstash]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-ml]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-monitoring]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-rollup]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-security]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-sql]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-upgrade]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] loaded module [x-pack-watcher]
[2019-01-19T15:38:38,493][INFO ][o.e.p.PluginsService     ] [node-HK143] no plugins loaded
[2019-01-19T15:38:44,393][INFO ][o.e.x.s.a.s.FileRolesStore] [node-HK143] parsed [0] roles from file [/usr/local/apps/elasticsearch/config/roles.yml]
[2019-01-19T15:38:45,263][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [node-HK143] [controller/17822] [Main.cc@109] controller (64 bit): Version 6.5.4 (Build b616085ef32393) Copyright (c) 2018 Elasticsearch BV
[2019-01-19T15:38:45,812][DEBUG][o.e.a.ActionModule       ] [node-HK143] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-01-19T15:38:46,074][INFO ][o.e.d.DiscoveryModule    ] [node-HK143] using discovery type [zen] and host providers [settings]
[2019-01-19T15:38:47,188][INFO ][o.e.n.Node               ] [node-HK143] initialized
[2019-01-19T15:38:47,188][INFO ][o.e.n.Node               ] [node-HK143] starting ...
[2019-01-19T15:38:47,379][INFO ][o.e.t.TransportService   ] [node-HK143] publish_address {172.31.xx.xx:9300}, bound_addresses {0.0.0.0:9300}
[2019-01-19T15:38:47,401][INFO ][o.e.b.BootstrapChecks    ] [node-HK143] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2019-01-19T15:38:50,468][INFO ][o.e.c.s.MasterService    ] [node-HK143] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {node-HK143}{0-iZEA3xQwiCW5nDjvkhcQ}{qNydEABaR7ihioJBuSbJcw}{172.31.216.34}{172.31.xx.xx:9300}{ml.machine_memory=8202706944, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2019-01-19T15:38:50,473][INFO ][o.e.c.s.ClusterApplierService] [node-HK143] new_master {node-HK143}{0-iZEA3xQwiCW5nDjvkhcQ}{qNydEABaR7ihioJBuSbJcw}{172.31.xx.xx}{172.31.216.34:9300}{ml.machine_memory=8202706944, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {node-HK143}{0-iZEA3xQwiCW5nDjvkhcQ}{qNydEABaR7ihioJBuSbJcw}{172.31.xx.xx}{172.31.216.34:9300}{ml.machine_memory=8202706944, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-01-19T15:38:50,502][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node-HK143] publish_address {172.31.216.34:9200}, bound_addresses {0.0.0.0:9200}
[2019-01-19T15:38:50,503][INFO ][o.e.n.Node               ] [node-HK143] started
[2019-01-19T15:38:50,531][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [node-HK143] Failed to clear cache for realms [[]]
[2019-01-19T15:38:50,604][INFO ][o.e.g.GatewayService     ] [node-HK143] recovered [0] indices into cluster_state
[2019-01-19T15:38:50,924][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.watches] for index patterns [.watches*]
[2019-01-19T15:38:50,994][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2019-01-19T15:38:51,108][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.watch-history-9] for index patterns [.watcher-history-9*]
[2019-01-19T15:38:51,153][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
[2019-01-19T15:38:51,214][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
[2019-01-19T15:38:51,255][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
[2019-01-19T15:38:51,292][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2019-01-19T15:38:51,328][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node-HK143] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
[2019-01-19T15:38:51,490][INFO ][o.e.l.LicenseService     ] [node-HK143] license [e385b812-7f2f-4762-9852-d45739bf87e7] mode [basic] - valid

TransportService ] [node-HK143] publish_address {172.31.xx.xx:9300}

SecurityNetty4HttpServerTransport] [node-HK143] publish_address {172.31.xx.xx:9200}

8. 测试ES 的启动是否正常。如果有如下返回结果，表明启动正常

# curl http://127.0.0.1:9200
{
  "name" : "node-xx143",
  "cluster_name" : "xx_xxx",
  "cluster_uuid" : "Tsn1S5HnQiaCeFwrOzJmSA",
  "version" : {
    "number" : "6.5.4",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "d2ef93d",
    "build_date" : "2018-12-17T21:17:40.758843Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Everything is working well

一步步建立 ELK log分析平台 --- elasticsearch 6.5.4 安装

1. 从官网下载6.5.4

2. 修改配置：#vi config/elasticsearch.yml

3. 修改/etc/sysctl.conf

4.修改文件/etc/security/limits.conf

5. 修改 /etc/security/limits.d/20-nproc.conf

6. 添加 es 用户

7. 启动ES

8. 测试ES 的启动是否正常。如果有如下返回结果，表明启动正常

猜你喜欢