Hadoop 3.1.1 - 概述 - 总览

Apache Hadoop 3.1.1

和之前发布的 3.0.X 版本线相比,Apache Hadoop 3.1.1 吸收了许多重要的改进。

总览

建议用户阅读完整的版本说明。本文提供了对主要变动的总览。

以下是对主要功能和改进的简短描述。

Yarn Service 框架为在YARN原生环境里长时间运行的服务,提供了一流的支持和接口。

简言之,它扮演了容器编排系统的角色,统一管理 Yarn 上运行的容器化服务。它同时支持 Docker 容器和传统基于进程的 Yarn 容器。

详情请阅用户文档

基于 Yarn 的一流的 GPU 调度和(对Docker和非Docker容器的)隔离。

详情请阅用户文档

基于 Yarn 的一流的 FPGA 调度和(对Docker和非Docker容器的)隔离。

详情请阅用户文档

Support more expressive placement constraints in YARN. Such constraints can be crucial for the performance and resilience of applications, especially those that include long-running containers, such as services, machine-learning and streaming workloads.

For example, it may be beneficial to co-locate the allocations of a job on the same rack (affinity constraints) to reduce network costs, spread allocations across machines (anti-affinity constraints) to minimize resource interference, or allow up to a specific number of allocations in a node group (cardinality constraints) to strike a balance between the two. Placement decisions also affect resilience. For example, allocations placed within the same cluster upgrade domain would go offline simultaneously.

See the user documentation for more details.

Support administrators to specify absolute resources (X Memory, Y VCores, Z GPUs, etc.) to a queue instead of providing percentage based values. This provides better control for admins to configure required amount of resources for a given queue.

See the user documentation for more details.

Provided storage allows data stored outside HDFS to be mapped to and addressed from HDFS. It builds on heterogeneous storage by introducing a new storage type, PROVIDED, to the set of media in a DataNode.

See the user documentation for more details.

上手

本文档包含你开始使用 Hadoop 所需的所有信息。请从【概述 - 单节点安装】开始,它展示了如何安装一个单机上运行的 Hadoop。然后,请移步【概述 - 集群安装】,学习如何安装多机上运行的 Hadoop。

猜你喜欢

转载自www.cnblogs.com/shishaochen/p/9634992.html