前言
在Hue上面测试一个很简单的Sqoop
import
--connect
jdbc:mysql://10.169.xx.xxx/test
--username
root
--password
root
--as-textfile
--columns
id,name,english,chinese,math
--table
exam
--target-dir
hdfs://master02:8020/tmp/test
--check-column
id
--incremental
append
-m
1
这个代码是在本地测试通过的，我本认为在Hue上可以很轻松的通过，没想到直接报错：
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
一看这个bug，刚开始觉得很简单，只要Sqoop获取到jdbc驱动就可以了，可是这个jdbc驱动应该往哪里放啊？还有为什么本地测试竟然可以，而Hue就不行呢？郁闷ing。。。
究竟往哪里放JDBC驱动包呢
- 往每个节点的/CDH/lib/sqoop/lib下面都放
- 往每个节点/var/lib/sqoop下面都放
- 往HDFS的/user/oozie/share/lib/lib_timstamp/sqoop目录下放
不对啊，放在/user/oozie/share/lib/lib_timstamp/sqoop目录下后得重启oozie服务啊
终于成功了…
后记
其实就两步：
1. 将mysql-connector-java.jar上传到hdfs：/user/oozie/share/lib/lib_timstamp/sqoop
2, 重启oozie服务。
还有，在hue的页面上无论出现什么错，总是报如下错误迷惑人：
>>> Invoking Sqoop command line now >>>
19:51:10.724 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
<<< Invocation of Sqoop command completed <<<
No child hadoop job is executed.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
Caused by: java.lang.SecurityException: Intercepted System.exit(1)
at org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
at java.lang.Runtime.exit(Runtime.java:107)
at java.lang.System.exit(System.java:971)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:214)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:199)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
... 16 more
Intercepting System.exit(1)
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice2/user/xuec7/oozie-oozi/0000000-191219193920246-oozie-oozi-W/sqoop-213e--sqoop/action-data.seq
19:51:10.778 [main] INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new compressor [.deflate]
Stopping AM
19:51:10.801 [main] INFO org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl - Waiting for application to be successfully unregistered.
Callback notification attempts left 0
Callback notification trying http://Utility02:11000/oozie/callback?id=0000000-191219193920246-oozie-oozi-W@sqoop-213e&status=FAILED
Callback notification to http://Utility02:11000/oozie/callback?id=0000000-191219193920246-oozie-oozi-W@sqoop-213e&status=FAILED succeeded
Callback notification succeeded
真的蛋疼啊，我在这个上面浪费了太多的时间
后知后觉
忽然在Hadoop实操的微信公众号里搜了一下Hue，才发现这个公众号早就回答了这个问题，以后多多学习
