Ambari 2.7.3 源码构建与安装问题

目录

简介:

安装环境:

构建工具安装:

源码构建与安装: 

构建问题:

参考:


简介:

Apache Ambari 项目主要通过开发软件用于提供、管理和监控Hadoop集群,来使得管理Hadoop集群更简单。Ambari通过使用RESTful API 提供了一个直观的、方便使用的Hadoop管理web UI。(官网链接

安装环境:

  • Centos:7.6
  • Mysql:5.7 (注意不建议使用Mysql8.0  参考链接

构建工具安装:

由于下载的Ambari是源码(下载链接),因此需要构建后安装使用,构建过程需要安装一些工具(参考链接)。

  •  JDK8(Ambari 2.6 and below can be compiled with JDK 7, from Ambari 2.7, it can be compiled with at least JDK 8)

   #安装完后查看版本信息

# java -version   
# mvn -v
  • Python 2.7 (Ambari 2.7 or later require Python 2.7 as minimum supported version)
#  python -V
  • Python setuptools:

    for Python 2.6: Download setuptools and run:

# sh setuptools-0.6c11-py2.6.egg

   for Python 2.7:  Download setuptools and run:

# sh setuptools-0.6c11-py2.7.egg
  • rpmbuild (rpm-build package)
# yum install -y rpm-build
  • g++ (gcc-c++ package)
# yum install -y gcc-c++
  • Node.js 

    这里需要官网(链接)下载(Linux Binaries (x64))安装,然后手动配置环境变量和路径;

# vim /etc/profile

 添加下面两行:
 export NODEJS=/***/***/node-v10.16.0-linux-x64      #解压NodeJS的路径
 export PATH=:$NODEJS/bin:$PATH

# source /etc/profile

   配置完后分别查看node 版本和npm 版本

# node -v

# npm -v
  • Brunch

   使用 npm 安装 Brunch

# npm install -g brunch

   安装完后检查版本信息:

# brunch -v
  • 修改 /etc/profile 文件, 设置JVM的参数,不然在编译时候很容易出现 OutOfMemoryError。 

   打开 /etc/profile (# vim  /etc/profile) 在最后一行添加如下代码:

export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true"

源码构建与安装: 

参考连接

1.下载源码

选择下载镜像(链接

# wget https://mirrors.tuna.tsinghua.edu.cn/apache/ambari/ambari-2.7.3/apache-ambari-2.7.3-src.tar.gz
# tar xfvz apache-ambari-2.7.3-src.tar.gz
# cd apache-ambari-2.7.3-src
# mvn versions:set -DnewVersion=2.7.3.0.0
 
# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.7.3.0.0
# popd

2.构建源码

# mvn -B clean install rpm:rpm -DnewVersion=2.7.3.0.0 -DbuildNumber=4295bb16c439cbc8fb0e7362f19768dde1477868 -DskipTests -Dpython.ver="python >= 2.6"

注意:这个过程一般花费时间比较长,由于下载依赖包比较费时间,可能需要花费一两天,建议分模块构建,其中也会遇到很多问题,遇到具体问题建议参考链接1链接2,这里在后面构建问题列出几个遇到的问题。

3.Ambari Server 安装

当构建成功后在 ambari-server/target/rpm/ambari-server/RPMS/noarch/ 目录或者 ambari-server/target/rpm/ambari-server/RPMS/x86_64/下会生成  ambari-server-2.7.3.0-0.x86_64.rpm 

# yum install ambari-server*.rpm

4.配置并启动Ambari Server

运行 ambari-server setup 命令 并且根据屏幕的命令提示一步步执行,配置Ambari Server, Database, JDK, LDAP和其他选项:

ambari-server setup

配置过程如下:

# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? n
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (***): 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
.
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

注意:这里需要根据提示的目录/home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql ,将此文件导入mysql(进入mysql 创建数据库后, source /home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql ),完成mysql的表的配置。

配置成功后启动 Ambari Server:

# ambari-server start

5.在所有主机安装并启动Ambari Agent 

在构建成功后,在/home/***/apache-ambari-2.7.3-src/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/  目录下会生成ambari-agent-2.7.3.0-0.x86_64.rpm 文件,将此文件拷贝到需要安装的主机,安装Ambari Agent ,

# yum install ambari-agent*.rpm

安装完成后配置主机地址,打开/etc/ambari-agent/conf/ambari-agent.ini  (vim /etc/ambari-agent/conf/ambari-agent.ini )文件:

[server]
#配置 Ambari-Server 安装的主机
hostname=172.16.18.103    
url_port=8440
secured_url_port=8441
connect_retry_delay=10
max_reconnect_retry_delay=30

配置完后启动 ambari-agent 

# ambari-agent start

6.通过 Ambari Web UI 部署集群

在浏览器  输入 http://<ambari-server-host>:8080.

输入用户名: admin 密码:admin

如果显示如下页面说明配置成功.

构建问题:

问题1.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /home/***/Apache_Ambari/apache-ambari-2.7.3-src/target/rat.txt -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

解决:

mvn 命令中增加 :-Drat.skip=true

问题2.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failοnerrοr="true" dir="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:325 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-metrics-host-monitoring
解决:

这个问题出现一般是因为之前下载某个软件超时,所以需要手动下载前面超时下载的jar包 ,放在下载时候指定的路径。

问题3.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project ambari-server: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-server

解决:

这是因为 pom.xml 文件指定的编译器版本和Jvm运行版本,与安装环境安装的JAVA和JVM版本不一致,本机安装了Java版本为1.8 ,所以需要修改pom.xml 文件中相应版本一致,修改如下部分:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.2</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
</plugin>

 问题4.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: The plugin com.github.eirslett:frontend-maven-plugin:1.4 requires Maven version 3.1.0 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :ambari-web

解决:

maven 版本太低,需要手动下载新版本的Maven,并配置环境变量。

问题5.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (phoenix_download) on project ambari-metrics-timelineservice: An Ant BuildException has occured: java.net.SocketException: Connection reset
[ERROR] around Ant part ...<get usetimestamp="true" src="http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz" dest="/home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/phoenix.tar.gz"/>... @ 5:301 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download Phoenix.xml
[ERROR] -> [Help 1]
解决:

这个是由于在构建ambari-metrics-timelineservice 这个模块的时候,下载phoenix-5.0.0.3.0.0.0-1634.tar.gz  超时,

一方面通过指定链接手动下载该文件 ,http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz,并按照要求修改名字后(phoenix.tar.gz)放在 /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/  路径下。

另一方面修改mvn库的镜像(参考5):

编辑mvn的 settings.xml文件,一般在 ~/.m2/settings.xml 目录或者,mvn 安装目录下 home/Maven/apache-maven-3.6.1/conf/settings.xml ,添加镜像http://repo.hortonworks.com/content/groups/public/  和 http://nexus-private.hortonworks.com/nexus/content/groups/public 。

<settings>
         <mirrors>
            <mirror>
               <id>public</id>
               <mirrorOf>*</mirrorOf>
               <url>http://nexus-private.hortonworks.com/nexus/content/groups/public</url>
            </mirror>
         </mirrors>
</settings>

问题6.

[INFO] Downloading http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz  to /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

[get] Getting: http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz 
[get] To: /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

解决:

这和问题五一样,是由于下载文件比慢,卡在这里,解决方法是手动去指定连接下载文件,然后修改名字后放在指定目录下。

参考:

1. https://www.cnblogs.com/barneywill/p/10264135.html

2. https://blog.csdn.net/ZhouyuanLinli/article/details/88963289

3. https://blog.csdn.net/slx_2011/article/details/45038089

4.https://blog.csdn.net/guoyachuan01/article/details/39500575

5.https://community.hortonworks.com/questions/157149/apache-ambari-error-during-maven-build-missing-ins.html

猜你喜欢

转载自blog.csdn.net/abcdu1/article/details/90710851
今日推荐