Hive中创建lzo表,插入lzo表

1,创建表的时候指定为lzo格式

CREATE EXTERNAL TABLE foo (
     columnA string,
     columnB string
) PARTITIONED BY (date string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY "\t"
STORED AS
INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat"
OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
LOCATION '/path/to/hive/tables/foo';

2,对于已经创建好的表,可以使用alter语句,修改为lzo存储格式。
ALTER TABLE foo
SET FILEFORMAT 
INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat"
OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat";


3,当使用insert语句往lzo表里插入数据时,需要加入下面两个参数:

SET hive.exec.compress.output=true;
SET mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;
--------------------- 
转自:https://blog.csdn.net/wisgood/article/details/44937147 

猜你喜欢

转载自blog.csdn.net/u010002184/article/details/88573318
LZO