搭建ELK日志系统(一)

先安装elasticsearch

1、确保jdk已经安装,这里省略

2、参考网址 https://www.elastic.co/guide/en/elasticsearch/reference/6.0/deb.html 可知安装方法有多种,我选择的是以下方法

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz.sha512
shasum -a 512 -c elasticsearch-7.5.0-linux-x86_64.tar.gz.sha512 
tar -xzf elasticsearch-7.5.0-linux-x86_64.tar.gz
cd elasticsearch-7.5.0/ 

先进入到除了root以外的目录,找一个你喜欢的目录(一般是放软件安装包的),按照这个步骤一步步的来安装,第三步的时候我遇到了 -bash: shasum: command not found 这个问题,通过执行 yum install perl-Digest-SHA 解决。

安装成功之后,启动es,进到es的安装目录,执行 ./bin/elasticsearch,报错

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
output:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# logs/hs_err_pid21335.log
error:
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
	at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:118)
	at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:86)
	at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:59)
	at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:92)

无法分配内存,cat  conf/jvm.options 里看到的 -Xms1g -Xmx1g,我把他们都改为256m,然后再启动,换了个报错信息

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
[2019-12-06T11:07:54,035][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [iZuf62rnwcgp54evv9uoo0Z] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.0.jar:7.5.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
	at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.5.0.jar:7.5.0]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.0.jar:7.5.0]
	... 6 more

这个是因为es默认不能用root用户启动,那下面我们去创建一个新的用户,和一个新的用户组

新增用户:

useradd elasticsearch
passwd elasticsearch

新增用户组:

groupadd elasticsearch

然后我 su elasticsearch 切换用户,然后去执行 ./bin/elasticsearch 报错

Error: Could not find or load main class org.elasticsearch.tools.java_version_checker.JavaVersionChecker

还记得我上面最开始的时候 第一个加红色的地方吗?一开始我是在root里安装es的,然后启动就会报这个错,所以我上面说不要在root目录里安装;然后我root用户执行

mv elasticsearch-7.5.0 /home/elasticsearch/ 命令把es包移到了home目录下,再启动es,报错

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
Exception in thread "main" java.nio.file.AccessDeniedException: /home/elasticsearch/elasticsearch-7.5.0/config/jvm.options
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
	at java.nio.file.Files.newInputStream(Files.java:152)
	at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:62)

这个是因为没权限,我上面新增用户和用户组之后,没有给用户授权,root用户执行

chown -R elasticsearch:elasticsearch  /home/elasticsearch/ 即可,再次用elasticsearch用户启动es,成功

新开一个窗口,执行curl http://localhost:9200/ 测试一下,出现下面json格式的信息,说明启动成功了

{
  "name" : "iZuf62rnwcgp54evv9uoo0Z",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "liYwoOgGQrirka3bSYrtiw",
  "version" : {
    "number" : "7.5.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "e9ccaed468e2fac2275a3761849cbee64b39519f",
    "build_date" : "2019-11-26T01:06:52.518245Z",
    "build_snapshot" : false,
    "lucene_version" : "8.3.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

至此,ES安装并启动成功,

启动:一般是后台 启动  ./elasticsearch -p /tmp/elasticsearch-pid -d

停止:jps | grep Elasticsearch 或者 cat /tmp/elasticsearch-pid && echo 找到PID,然后 kill -SIGTERM  PID。

但是你有没有发现,这里只能用http://localhost:9200/访问,用服务器内网和外网都不能访问,如果我们想在浏览器里用外网访问该怎么办呢?

vim /home/elasticsearch/elasticsearch-7.5.0/config/elasticsearch.yml

Network模块里

network.host: 0.0.0.0
network.publish_host: 你自己的服务器外网地址

重启,然后在阿里云安全组里打开9200这个端口,浏览器访问:http://外网:9200 即可访问

 期间遇到的问题:可参考这个,我遇到的问题和这一样。。。

https://blog.csdn.net/weixin_38361347/article/details/93386009

 然后还需要安装es的可视化插件es-head

git clone git://github.com/mobz/elasticsearch-head.git  (没有git命令的话 yum install npm)
cd elasticsearch-head
npm install   (没有npm命令的话 yum install npm)
npm run start  

我是在npm install这一步出现错误的,错误信息如下:

可以看到应该是没有bzip2命令,安装 yum install bzip2 

再次执行npm install,成功

然后启动:npm run start 

启动成功,测试一下 curl http://localhost:9100

是成功的,然后我们把阿里云安全组里加入 9100端口,浏览器里使用外网访问:

外网可以访问,但是右边集群健康值:是未连接的,我们重启下 es,没有报错,但是浏览器里我还是连接不上我的es,

后来百度查到可能是 跨域的问题,我在elasticsearch.yml里新增下面两条命令

#allow origin
http.cors.enabled: true
http.cors.allow-origin: "*"

重启es,打开浏览器ES控制面板,成功连接

至此为止,ES和ES的可视化插件就已经安装好了。

下篇文章继续安装Logstash

发布了165 篇原创文章 · 获赞 103 · 访问量 39万+

猜你喜欢

转载自blog.csdn.net/qq_33101675/article/details/103423280