azkaban3.43.0安装教程（git编译版）

一、环境要求

• mysql数据库
• linux --git

二、下载编译安装

1、从Git下载源码

cd /root
git clone https://github.com/azkaban/azkaban.git

2、下载完成后切换到azkaban目录下(gradlew 压缩获取tar包)

cd /root/azkaban

编译

./gradlew distTar

显示 BUILD SUCCESSFUL 压缩成功

注：所有压缩后的tar包都在各目录下的/build/distributions目录下

例如：/root/azkaban/azkaban-web-server的tar包/build/distributions 目录下为：azkaban-web-server的tar包

3、复制所有tar包到azkaban目录下

cp /root/azkaban/azkaban-*/build/distributions/*.tar.gz  /root/azkaban

cd /root/azkaban

tar zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz
tar zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
tar zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz

4、导入表到Mysql数据库

（sql文件为：/root/azkaban/azkaban-db/build/sql目录下create-all-sql-3.46.0-2-gf9ec3aa.sql文件） 

5、生成keystore文件

cd /root/keystore

keytool -keystore keystore -alias jetty -genkey -keyalg RSA

输入密钥库口令:（输入keystore）  
再次输入新口令: （输入keystore）
您的名字与姓氏是什么?
  [Unknown]:  
您的组织单位名称是什么?
  [Unknown]:  
您的组织名称是什么?
  [Unknown]:  
您所在的城市或区域名称是什么?
  [Unknown]:  
您所在的省/市/自治区名称是什么?
  [Unknown]:  
该单位的双字母国家/地区代码是什么?
  [Unknown]:  CN
CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=CN是否正确?
  [否]:  y

输入 <jetty> 的密钥口令
	(如果和密钥库口令相同, 按回车): （回车）

6、配置conf/azkaban.properties：

如果Azkaban WebServer下面没有conf目录，将azkaban-solo-web 下的conf目录拷贝过来

# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/root/azkaban/azkaban-web-server-3.43.0/conf/azkaban-users.xml
# Loader for projects
executor.global.properties=/root/azkaban/azkaban-web-server-3.43.0/conf/global.properties
azkaban.project.dir=projects
project.temp.dir=temp
trigger.plugin.dir=/root/azkaban/azkaban-web-server-3.43.0/plugins/triggers
#database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=3306
mysql.host=192.168.66.170
mysql.database=azkaban
mysql.user=root
mysql.password=sql_9879
mysql.numconnections=100
# Velocity dev mode
velocity.dev.mode=false
# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8666
jetty.port=8081
jetty.keystore=web/keystore
jetty.password=123456
jetty.keypassword=123456
jetty.truststore=web/keystore
jetty.trustpassword=123456
jetty.excludeCipherSuites=SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,SSL_DHE_DSS_WITH_DES_CBC_SHA,SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA

#jetty.use.ssl=false
#jetty.maxThreads=25
#jetty.port=8081
# Azkaban Executor settings
executor.port=12321
# mail settings
[email protected] 
mail.host=smtp.163.com
[email protected]
[email protected]
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links. 
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081

lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
#azkaban.jobtype.plugin.dir=plugins/jobtypes

5.修改azkabanazkaban-web-server-3.43.0文件

（1）复制conf目录到/root/azkaban/azkaban-exec-server-3.43.0/目录下

cd /root/azkaban
cp conf /root/azkabanazkaban-web-server-3.43.0/

（2）azkaban-web-server 配置azkaban.properties文件

cd /root/azkaban/azkaban-exec-server-3.43.0/conf

内容如下（注将时区：default.timezone.id 改为Asia/Shanghai）：

#Azkaban
default.timezone.id=Asia/Shanghai

# Azkaban JobTypes Plugins
azkaban.jobtype.plugin.dir=/root/azkaban/azkaban-exec-server-3.43.0/plugins/azkaban-jobtype-2.5.0

#Loader for projects
executor.global.properties=/root/azkaban/azkaban-exec-server-3.43.0/conf/global.properties
azkaban.project.dir=projects
azkaban.execution.dir=executions
project.temp.dir=temp

database.type=mysql
mysql.port=3306
mysql.host=192.168.66.170
mysql.database=azkaban
mysql.user=root
mysql.password=sql_9879
mysql.numconnections=100

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

如果启动时报错

报错：Exception: java.lang.StackOverflowError thrown from the UncaughtExceptionHandler in thread “main”

解决方式：在Azkaban-web-server 的conf目录下创建文件夹conf/log4j.properties：

log4j.rootLogger=INFO,C
log4j.appender.C=org.apache.log4j.ConsoleAppender
log4j.appender.C.Target=System.err
log4j.appender.C.layout=org.apache.log4j.PatternLayout
log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

7.访问并输入

账号：azkaban 密码：azkaban

https://192.168.66.194:8666

(账号密码设置修改在 /root/azkaban/azkaban-web-server-3.43.0/conf/azkaban-users.xml中配置)

<azkaban-users>
	<user username="azkaban" password="azkaban" roles="admin" groups="azkaban" />
	<user username="metrics" password="metrics" roles="metrics"/>
	<user username="admin" password="admin" roles="admin,metrics" />
	<role name="admin" permissions="ADMIN" />
	<role name="metrics" permissions="METRICS"/>
</azkaban-users>

出现以下页面部署成功：

7、 Azkaban Executor Server 安装和配置

进入azkaban-executor-server目录，将azkaban-web-server下的conf目录拷贝到该目录下

1)配置executor端的azkaban.properties

#Azkaban
default.timezone.id=Asia/Shanghai

# Azkaban JobTypes Plugins
azkaban.jobtype.plugin.dir=/root/azkaban/azkaban-exec-server-3.43.0/plugins/azkaban-jobtype-2.5.0

#Loader for projects
executor.global.properties=/root/azkaban/azkaban-exec-server-3.43.0/conf/global.properties
azkaban.project.dir=projects
azkaban.execution.dir=executions
project.temp.dir=temp

database.type=mysql
mysql.port=3306
mysql.host=192.168.66.170
mysql.database=azkaban
mysql.user=root
mysql.password=sql_9879
mysql.numconnections=100

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

6.启动 Web Server和 Executor Server

sh bin/azkaban-executor-start.sh 

错误解决

运行一个demo，发现报错 Missing required property 'azkaban.native.lib'

在官网中有这样一段话：
Plugin Configurations
Execute-As-User
With a new security enhancement in Azkaban 3.0, Azkaban jobs can now run as the submit user or the user.to.proxy of the flow by default. This ensures that Azkaban takes advantage of the Linux permission security mechanism, and operationally this simplifies resource monitoring and visibility. Set up this behavior by doing the following:-
Execute.as.user is set to true by default. In case needed, it can also be configured to false in azkaban-plugin’s commonprivate.properties
Configure azkaban.native.lib= to the place where you are going to put the compiled execute-as-user.c file (see below)
Generate an executable on the Azkaban box for azkaban-common/src/main/c/execute-as-user.c. it should be named execute-as-user Below is a sample approach
scp ./azkaban-common/src/main/c/execute-as-user.c onto the Azkaban box
run: gcc execute-as-user.c -o execute-as-user
run: chown root execute-as-user (you might need root privilege)
run: chmod 6050 execute-as-user (you might need root privilege)
但是在最新的版本里面没有这个 execute-as-user.c 文件了

解决方法：
a) 先关闭两个服务
b) 在编译好的  azkaban-solo-server 中将 plugins/jobtypes/commonprivate.properties 连带目录直接拷贝到
azkaban-exec-server 目录中，并在 commonprivate.properties 添加一条 azkaban.native.lib=false
commonprivate.properties 中有两条记录：
execute.as.user=false
azkaban.native.lib=false