基于hadoop 0.19.0 + jdk 1.6 + eclipse 3.3.2 开发环境配置

今天配置了一下hadoop0.19.0开发环境,

1.首先安装JDK1.6(因为hadoop 0.19.0 是需要使用 jdk1.6 ),安装Eclipse 3.3.2
2.修改eclipse.ini
增加粗体:
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-vm
C:\Program Files\Java\jdk1.6.0_12\bin\javaw.exe

-vmargs
-Xms256m
-Xmx512m
3.下载hadoop0.19.0,下载完成以后将hadoop-0.19.0\contrib\eclipse-plugin\hadoop-0.19.0-eclipse-plugin.jar复制到eclipse的plugins目录
4.启动eclipse
5.选择菜单Windows---》Open Perspective---》Other
选择Map/Reduce OK。
6.Windows---》Preperence---》Hadoop Map/Reduce---》Hadoop installation directory,选择hadoop安装路径。
7.新建Map/Reduce项目
8.新建Java程序WordCount,然后将hadoop-0.19.0\src\examples\org\apache\hadoop\examples\WordCount.java复制到你的代码中,修改包名,修改代码,
增加一行:
conf.set("hadoop.job.ugi", "hadoop,hadoop");
如果不增加这行代码,会报:
09/02/10 21:28:46 WARN fs.FileSystem: uri=hdfs://hadoop2.xunjienet.com:9000/
javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, 

9.右键Run as ---》Run On hadoop
10.可以新建一个hadoop server location,主要是设置Map/Reduce Master和DFS Master的主机名和端口号,一般选择相同的主机名,Map/Reduce Master和DFS Master的端口号是9001,9000.

和服务器的hadoop-site.xml配置相关:

<property>
<name>fs.default.name</name>
<value>hdfs://hadoop2.xunjienet.com:9000</value>
<description>The name of the default file system. Either the literal string 
"local" or a host:port for DFS.</description>
</property>
<property>
<name>mapred.job.tracker</name>
<value>hadoop2.xunjienet.com:9001</value>
<description>The host and port that the MapReduce job tracker runs at. If 
"local", then jobs are run in-process as a single map and reduce task.</description>
</property>

猜你喜欢

转载自androidssh.iteye.com/blog/810620