There is a big pit in building the hadoop environment: the version mismatch of the hadoop technology stack .
After building hive, there is no problem with running tests separately with hdfs. After building sqoop, it is no problem to use sqoop and hdfs together.
The three are used together, and the following command is executed to import the table in mysql into hive:
sqoop import --connect 'jdbc:mysql://192.168.174.131:3306/ecp?useUnicode=true&characterEncoding=utf-8' --username root --password 123456 --table ecp_user -m 2 --hive-import
The result is an error:
ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
Worrying about the configuration problem, I reinstalled hive and sqoop according to different tutorials, but there is still no solution.
There is only one answer from Du Niang and Google search, suggesting that the libthrift-*.jar in the lib folder under sqoop is inconsistent with the libthrift-*.jar in the lib folder under hive, and the libthrift-0.9.3 in hive. Copy the jar file to the lib folder under sqoop and execute it again, but still report an error.
Finally, I suspect that it is a version problem. sqoop has been using 1.4.7. At the beginning, hive2.3.2 and mysql-connector-java-5.1.46-bin.jar reported an error; changed to hive2.2.0 and mysql-connector-java-5.1. 46-bin.jar, report an error; change to hive2.2.0 and mysql-connector-java-5.1.39-bin.jar, report an error; change to hive2.1.1 and mysql-connector-java-5.1.39-bin.jar , run successfully.
I hope to reduce the time for future people to spend on setting up the environment