在Ubtuntu下安装hadoop集群,安装java环境
主要参考 http://www.linuxidc.com/Linux/2015-01/112030.htm
1、 首先从官网上下载对应版本的jdk,我下载的是jdk-7u79-linux-x64.tar.gz,放到home主目录下
2、 打开终端 Ctrl+Alt+T,创建java的目录,输入密码
master@ubuntu:~$ sudo mkdir /usr/local/java
[sudo] password for master:
3、 把压缩包复制到java目录下:
master@ubuntu:~$ cp jdk-7u79-linux-x64.tar.gz/usr/local/java
可能会提示权限不足,前面加上sudo即可,
cp: cannot create regular file '/usr/local/java/jdk-7u79-linux-x64.tar.gz': Permission denied
master@ubuntu:~$ sudo cp jdk-7u79-linux-x64.tar.gz /usr/local/java
4、 定位到java目录下,
master@ubuntu:~$ cd /usr/local/java
5、 解压压缩包,等待完成
master@ubuntu:/usr/local/java$ sudo tar xvf jdk-7u79-linux-x64.tar.gz
可以删除压缩包,
master@ubuntu:/usr/local/java$ sudo rm jdk-7u79-linux-x64.tar.gz
6、 编辑jdk环境变量,用全局设置方法,它是是所有用户的共用的环境变量
master@ubuntu:/usr/local/java$ sudo gedit ~/.bashrc
7、 打开文件,在最后添加下面的代码,注意更改自己jdk的版本号,后面不要多空格
export JAVA_HOME=/usr/local/java/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
8、 查看是否安装成功
master@ubuntu:~$ java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)
http://www.cnblogs.com/kinglau/p/3794433.html
安装Hadoop过程如下:
1、下载hadoop,放在home目录,解压,在local下面创建hadoop文件夹
master@ubuntu:~$ sudo cp hadoop-2.5.2.tar.gz /usr/local //复制到local下面
master@ubuntu:~$ sudo tar -zxf hadoop-2.5.2.tar.gz -C /usr/local //解压
master@ubuntu:~$ cd /usr/local/
master@ubuntu:/usr/local$ sudo mv ./hadoop-2.5.2/ ./hadoop //改名
master@ubuntu:/usr/local$ sudo chown -R master ./hadoop //赋予文件夹权限
2、配置hadoop,输入java环境
master@ubuntu:~$ sudo gedit /usr/local/hadoop/etc/hadoop/hadoop-env.sh
[sudo] password for master:
master@ubuntu:~$ sudo gedit ~/.bashrc //上面配置过java环境,打开查看java的路径
找到JAVA_HOME变量,修改此变量如下
export JAVA_HOME=/usr/local/java/jdk1.7.0_79
3、利用Wordcount,测试hadoop
master@ubuntu:~$ cd /usr/local/hadoop
master@ubuntu:/usr/local/hadoop$ sudo mkdir input //当前目录新建一个input文件夹
master@ubuntu:/usr/local/hadoop$ sudo cp README.txt input //copy txt文件进去
master@ubuntu:/usr/local/hadoop$ bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.5.2-sources.jar org.apache.hadoop.examples.WordCount input output //根据自己的hadoop版本修改
成功后返回如下:
17/03/23 20:26:48 INFO mapreduce.Job: Counters: 33
File System Counters
FILE: Number of bytes read=547402
FILE: Number of bytes written=1013492
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=31
Map output records=179
Map output bytes=2055
Map output materialized bytes=1836
Input split bytes=104
Combine input records=179
Combine output records=131
Reduce input groups=131
Reduce shuffle bytes=1836
Reduce input records=131
Reduce output records=131
Spilled Records=262
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=48
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=240787456
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=1366
File Output Format Counters
Bytes Written=1326