Alluxio 1.6.1的安装部署

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/liuguangrong/article/details/78487932

Alluxio 1.6.1 下载

下载地址:https://www.alluxio.org/download
选择版本:Alluxio 1.6.1 (Nov 3 2017) Alluxio Distribution (pre-built for Hadoop 2.7)
Download
接着,你可以用如下命令解压下载包。取决于你下载的预编译二进制版本,你的文件名可能和下面有所不同。

$ tar -zxf alluxio-1.6.1-hadoop-2.7-bin.tar.gz -C /opt/bigdata/
$ cd alluxio-1.6.1-hadoop-2.7/

配置Alluxio

在开始使用Alluxio之前,我们需要配置它。大部分使用默认设置即可。
在${ALLUXIO_HOME}/conf目录下,根据模板文件创建conf/alluxio-env.sh配置文件。

# This file is an optional approach to configure Alluxio options by setting the
# following listed environment variables. Note that, setting this file will not
# affect jobs (e.g., Spark job or MapReduce job) that are using Alluxio client
# as a library. Alternatively, it is recommended to create alluxio-site.properties file,
# which supports all the configuration options provided by Alluxio
# (http://www.alluxio.org/documentation/en/Configuration-Settings.html),
# and is respected by both external jobs and Alluxio servers (or shell).
JAVA_HOME=/opt/modules/jdk1.8.0_121

# The directory where Alluxio deployment is installed. (Default: the parent directory of libexec/).
ALLUXIO_HOME=/opt/bigdata/alluxio-1.6.1-hadoop-2.7

# The directory where log files are stored. (Default: ${ALLUXIO_HOME}/logs).
# ALLUXIO_LOGS_DIR

# Hostname of the master.
ALLUXIO_MASTER_HOSTNAME=bigdata02

# This is now deprecated. Support will be removed in v2.0
# ALLUXIO_MASTER_ADDRESS

# The directory where a worker stores in-memory data. (Default: /mnt/ramdisk).
# E.g. On linux,  /mnt/ramdisk for ramdisk, /dev/shm for tmpFS; on MacOS, /Volumes/ramdisk for ramdisk
# ALLUXIO_RAM_FOLDER

# Address of the under filesystem address. (Default: ${ALLUXIO_HOME}/underFSStorage)
# E.g. "/my/local/path" to use local fs, "hdfs://localhost:9000/alluxio" to use a local hdfs
ALLUXIO_UNDERFS_ADDRESS=hdfs://bigdata01:8020/alluxio

在conf/alluxio-site.properties文件中将 alluxio.master.hostname更新为你打算运行Alluxio Master的机器主机名。如下是示例:

# Common properties
alluxio.master.hostname=bigdata02
alluxio.underfs.address=hdfs://bigdata01:8020/alluxio

# Security properties
# alluxio.security.authorization.permission.enabled=true
# alluxio.security.authentication.type=SIMPLE

# Worker properties
# alluxio.worker.memory.size=1GB
# alluxio.worker.tieredstore.levels=1
# alluxio.worker.tieredstore.level0.alias=MEM
# alluxio.worker.tieredstore.level0.dirs.path=/mnt/ramdisk

# User properties
# alluxio.user.file.readtype.default=CACHE_PROMOTE
# alluxio.user.file.writetype.default=MUST_CACHE
alluxio.keyvalue.enabled=true
alluxio.keyvalue.partition.size.bytes.max=256MB

alluxio.worker.tieredstore.levels=3
alluxio.worker.tieredstore.level0.alias=MEM
alluxio.worker.tieredstore.level0.dirs.path=/mnt/ramdisk
alluxio.worker.tieredstore.level0.dirs.quota=1GB
alluxio.worker.tieredstore.level0.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level0.watermark.low.ratio=0.7
alluxio.worker.tieredstore.level1.alias=SSD
alluxio.worker.tieredstore.level1.dirs.path=/opt/data/bigdata/alluxio
alluxio.worker.tieredstore.level1.dirs.quota=25GB
alluxio.worker.tieredstore.level1.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level1.watermark.low.ratio=0.7
alluxio.worker.tieredstore.level2.alias=HDD
alluxio.worker.tieredstore.level2.dirs.path=/data/bigdata/alluxio
alluxio.worker.tieredstore.level2.dirs.quota=200GB
alluxio.worker.tieredstore.level2.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level2.watermark.low.ratio=0.7

在conf/masters文件中配置Master的机器主机名。如下是示例:

bigdata02

在conf/workers文件中配置Worker的机器主机名。如下是示例:

bigdata01
bigdata03
bigdata04
bigdata05

验证Alluxio运行环境

在启动Alluxio前,我们要保证当前系统环境下Alluxio可以正常运行。我们可以通过运行如下命令来验证Alluxio的本地运行环境:

$ ./bin/alluxio validateEnv local

该命令将汇报在本地环境运行Alluxio可能出现的问题。如果你配置Alluxio运行在集群中,并且你想要验证所有节点的运行环境,你可以运行如下命令:

$ ./bin/alluxio validateEnv all

启动 Alluxio

现在,你可以启动 Alluxio:

$ ./bin/alluxio format
$ ./bin/alluxio-start.sh

为了确保Alluxio正在运行, 访问 http://:19999, 检查文件夹alluxio/logs下的日志, or 或者运行简单程序:

$ ./bin/alluxio runTests

参考资料

【1】 Alluxio官网:https://www.alluxio.org/

猜你喜欢

转载自blog.csdn.net/liuguangrong/article/details/78487932