Disclaimer: This article is a blogger original article, shall not be reproduced without the bloggers allowed. https://blog.csdn.net/wjandy0211/article/details/90229396

Cluster management guide describes how to use the Cloudera Cloudera Manager configuration and management of clustered enterprise deployment. Cloudera Enterprise Hadoop administrators to manage resources, hosting, high availability, and backup and restore configuration. Cloudera Manager management console is the primary tool used by administrators to monitor and manage the cluster. You can also Cloudera Manager API for cluster management tasks.

Performance Management

https://www.cloudera.com/documentation/enterprise/6/6.0/topics/cdh_admin_performance.html#cdh_performance

Disable the tuned Service

Ensure that the tuned service is started:
```
systemctl start tuned
```
Turn the tuned service off:
```
tuned-adm off
```
Ensure that there are no active profiles:
```
tuned-adm list
```
The output should contain the following line:
```
No current active profile
```

Shutdown and disable the tuned service:

systemctl stop tuned
systemctl disable tuned

Disabling Transparent Hugepages

To see whether transparent hugepages are enabled, run the following commands and check the output:

$ cat defrag_file_pathname
$ cat enabled_file_pathname

[always] never means that transparent hugepages is enabled.
always [never] means that transparent hugepages is disabled.

To disable transparent hugepages on reboot, add the following commands to the /etc/rc.d/rc.local file on all cluster hosts:

RHEL/CentOS 7.x:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

RHEL/CentOS 6.x

echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Setting the vm.swappiness Linux Kernel Parameter

To view your current setting for vm.swappiness, run:

cat /proc/sys/vm/swappiness

To set vm.swappiness to 1, run:

sudo sysctl -w vm.swappiness=1

Improving Performance in Shuffle Handler and IFile Reader

Best Practices for MapReduce Configuration

Send a heartbeat as soon as a task finishes

<property>
    <name>mapreduce.tasktracker.outofband.heartbeat</name>
    <value>true</value>
</property>

Reduce the interval for JobClient status reports on single node systems

<property>
    <name>jobclient.progress.monitor.poll.interval</name>
    <value>10</value>
</property>

Tune the JobTracker heartbeat interval

Tuning the minimum interval for the TaskTracker-to-JobTracker heartbeat to a smaller value may improve MapReduce performance on small clusters.

<property>
    <name>mapreduce.jobtracker.heartbeat.interval.min</name>
    <value>10</value>
</property>

Start MapReduce JVMs immediately

Map. Reduce .slowstart.completed.maps attribute specifies the proportion of mapping tasks must be completed before schedule any task Reduce tasks. For small jobs require quick turnaround, this value is set to 0 can improve performance; larger values (up to 50%) may be suitable for larger jobs.

<property>
    <name>mapred.reduce.slowstart.completed.maps</name>
    <value>0</value>
</property>

Decrease Reserve Space

Cloudera recommends reducing the root user block reservation from 5% to 1% for the DataNode volumes. To set reserved space to 1% with the tune2fs command:

# tune2fs -m 1 /dev/sde1

cloudera--CDH Performance Management