1.order by 和 sort by
1.order by
hive中的order by 语句和其他的sql方言中定义是一样的，其会对查询结果集执行一个全局排序，这也就是说会有一个所有的数据都通过一个reduce进行处理的过程，对于大数据集，这个过程将消耗很大的时间来执行。
2,sort by
hive增加了一个可供选择的方式，也就是sort by ,其实会在每个reduce中对数据进行排序，也就是执行一个局部排序过程。这可以保证每个reduce的输出数据都是有序的(但并非全局有效)。这样就可以提高后面进行的全局排序的效率了。对于这两种情况，语法区别仅仅是，一个关键字是order，另一个关键字是sort。用户可以指定任意期望进行排序的字段，并可以在字段后面加上asc关键字(默认)表示升序，desc关键字是降序排序
1，order by select * from zxz_5 order by nid desc,year(ntime); 结果: +-----------------------+------------+---------------+--------------+-------------+--------------+ | zxz_5.name | zxz_5.nid | zxz_5.phone | zxz_5.ntime | zxz_5.year | zxz_5.month | +-----------------------+------------+---------------+--------------+-------------+--------------+ | Cleon Haresnape | 100 | 625-338-3965 | 2014-12-04 | 2018 | 8 | | Blaine Fawdry | 99 | 477-109-9014 | 2012-07-14 | 2018 | 8 | | Zoe Adacot | 98 | 704-987-0702 | 2015-09-29 | 2018 | 8 | | Orlando Pallatina | 97 | 354-125-1208 | 2012-07-12 | 2018 | 8 | | Alyssa Prydden | 96 | 963-170-0545 | 2014-11-07 | 2018 | 8 | | Skipp Spurden | 95 | 747-133-1382 | 2012-03-15 | 2018 | 8 | | Auguste Kares | 94 | 230-184-3438 | 2014-03-13 | 2018 | 8 | | Ciro Arendt | 93 | 792-967-0588 | 2015-11-07 | 2018 | 8 | | Floyd Cano | 92 | 133-768-6535 | 2016-02-27 | 2018 | 8 | | Sayre Osbaldeston | 91 | 340-132-2361 | 2011-11-30 | 2018 | 8 | | Pearline Marcq | 90 | 200-835-9497 | 2016-02-10 | 2018 | 8 | | Zackariah Pietrusiak | 89 | 810-738-9846 | 2012-02-25 | 2018 | 8 | | Auguste Kobel | 88 | 562-494-1360 | 2012-02-29 | 2018 | 8 | | Jilly Eisikowitz | 87 | 431-355-2777 | 2017-02-18 | 2018 | 8 | | Wes Jaffrey | 86 | 848-465-5131 | 2016-02-14 | 2018 | 8 | | Casey Greenfield | 85 | 204-108-7707 | 2012-03-18 | 2018 | 8 | | Maurise Gallico | 84 | 546-158-7983 | 2011-12-21 | 2018 | 8 | | Cilka Heijne | 83 | 772-704-7366 | 2011-08-27 | 2018 | 8 | 2,sort by 在sort by之前我们还有配置属性: set mapreduce.job.reduces=2;//配置两个ruduce默认一个，不然sort by是没有用处的。 select * from zxz_5 sort by nid ，year(ntime) desc ; +-----------------------+------------+---------------+--------------+-------------+--------------+ | zxz_5.name | zxz_5.nid | zxz_5.phone | zxz_5.ntime | zxz_5.year | zxz_5.month | +-----------------------+------------+---------------+--------------+-------------+--------------+ | Reine Leyborne | 19 | 322-644-5798 | 2015-01-05 | 2018 | 8 | | Worth Ledbetter | 18 | 905-586-2348 | 2012-09-25 | 2018 | 8 | | Lloyd Nayshe | 15 | 366-495-5398 | 2014-06-28 | 2018 | 8 | | Field McGraith | 13 | 149-133-9607 | 2015-10-06 | 2018 | 8 | | Reube Preskett | 10 | 918-740-2357 | 2015-03-26 | 2018 | 8 | | Theressa Dowker | 8 | 864-330-9976 | 2012-09-26 | 2018 | 8 | | Claudetta Sanderson | 6 | 350-766-4559 | 2016-11-04 | 2018 | 8 | | Kevina Rawet | 4 | 955-643-0317 | 2012-01-05 | 2018 | 8 | | Gabriela Feldheim | 2 | 706-232-4166 | 2017-12-16 | 2018 | 8 | | Matias McGrirl | 1 | 945-639-8596 | 2011-11-24 | 2018 | 8 | | Cilka Heijne | 83 | 772-704-7366 | 2011-08-27 | 2011 | 8 | | Kyle Matheson | 63 | 153-149-2140 | 2011-07-03 | 2011 | 7 | | Lela Hinzer | 50 | 147-386-3735 | 2011-06-03 | 2011 | 6 | | Hoebart Deeth | 22 | 780-240-0213 | 2011-09-19 | 2011 | 9 |
2，sort by 和 distribute by
distribute by 控制 map的输出在reduer中是如何划分的，mapreduce job 中传输的所有数据都是按照键-值对的方式进行组织的，因此hive在将用户的查询语句转换成mapreduce job时，其必须在内部使用这个功能。默认情况下，MapReduce计算框架会依据map输入的键计算相应的哈希值，然后按照得到的哈希值将键-值对均匀分发到多个reducer中去，不过不幸的是，这也是意味着当我们使用sort by 时，不同reducer的输出内容会有明显的重叠，至少对于排序顺序而已只这样，即使每个reducer的输出的数据都有序的。如果我们想让同一年的数据一起处理，那么就可以使用distribute by 来保证具有相同年份的数据分发到同一个reducer中进行处理，然后使用sort by 来安装我们的期望对数据进行排序:
select * from zxz_5 z distribute by year(z.ntime) sort by z.id ; +-----------------------+--------+---------------+-------------+---------+----------+ | z.name | z.nid | z.phone | z.ntime | z.year | z.month | +-----------------------+--------+---------------+-------------+---------+----------+ | Matias McGrirl | 1 | 945-639-8596 | 2011-11-24 | 2018 | 8 | | Gabriela Feldheim | 2 | 706-232-4166 | 2017-12-16 | 2017 | 12 | | Gabriela Feldheim | 2 | 706-232-4166 | 2017-12-16 | 2018 | 8 | | Billy O'Driscoll | 3 | 660-841-7326 | 2017-02-17 | 2018 | 8 | | Billy O'Driscoll | 3 | 660-841-7326 | 2017-02-17 | 2017 | 2 | | Kevina Rawet | 4 | 955-643-0317 | 2012-01-05 | 2018 | 8 | | Patty Entreis | 5 | 571-792-2285 | 2013-06-11 | 2013 | 6 | | Patty Entreis | 5 | 571-792-2285 | 2013-06-11 | 2018 | 8 | | Claudetta Sanderson | 6 | 350-766-4559 | 2016-11-04 | 2018 | 8 | | Bentley Oddie | 7 | 446-519-0975 | 2016-05-02 | 2018 | 8 | | Theressa Dowker | 8 | 864-330-9976 | 2012-09-26 | 2018 | 8 | | Jenica Belcham | 9 | 347-248-4379 | 2011-05-02 | 2018 | 8 | | Reube Preskett | 10 | 918-740-2357 | 2015-03-26 | 2018 | 8 | | Mary Skeldon | 11 | 361-159-8710 | 2016-03-09 | 2018 | 8 | | Ethelred Divisek | 12 | 995-145-7392 | 2016-10-18 | 2016 | 10 | | Ethelred Divisek | 12 | 995-145-7392 | 2016-10-18 | 2018 | 8 | | Field McGraith | 13 | 149-133-9607 | 2015-10-06 | 2015 | 10 | | Field McGraith | 13 | 149-133-9607 | 2015-10-06 | 2018 | 8 | | Andeee Wiskar | 14 | 315-207-5431 | 2012-05-10 | 2018 | 8 | | Lloyd Nayshe | 15 | 366-495-5398 | 2014-06-28 | 2014 | 6 | | Lloyd Nayshe | 15 | 366-495-5398 | 2014-06-28 | 2018 | 8 | Distribute by 和 group by 在其控制着reducer是如何接受一行行数据进行处理这方面类似的，而sort by 则控制着reducer内的数据是如何进行排序的，需要注意的是，hive需求distribute by 语句写在sort by 语句前
3，cluster by
在之前列子中，year（ntime）用在distribute by 语句中，而nid列位于sort by语句中，如果这两个语句涉及到列的完全相同，而且采用的是升序的方式，那么就可以用到cluster by ，他等价于 distribute by 和sort by 是一个简写方式
select * from zxz_5 z cluster by z.nid ; (同似select * from zxz_5 z distribute by z.nid sort by z.nid ;) 结果: +-----------------------+--------+---------------+-------------+---------+----------+ | z.name | z.nid | z.phone | z.ntime | z.year | z.month | +-----------------------+--------+---------------+-------------+---------+----------+ | Matias McGrirl | 1 | 945-639-8596 | 2011-11-24 | 2018 | 8 | | Matias McGrirl | 1 | 945-639-8596 | 2011-11-24 | 2011 | 11 | | Gabriela Feldheim | 2 | 706-232-4166 | 2017-12-16 | 2017 | 12 | | Gabriela Feldheim | 2 | 706-232-4166 | 2017-12-16 | 2018 | 8 | | Billy O'Driscoll | 3 | 660-841-7326 | 2017-02-17 | 2018 | 8 | | Billy O'Driscoll | 3 | 660-841-7326 | 2017-02-17 | 2017 | 2 | | Kevina Rawet | 4 | 955-643-0317 | 2012-01-05 | 2012 | 1 | | Kevina Rawet | 4 | 955-643-0317 | 2012-01-05 | 2018 | 8 | | Patty Entreis | 5 | 571-792-2285 | 2013-06-11 | 2018 | 8 | | Patty Entreis | 5 | 571-792-2285 | 2013-06-11 | 2013 | 6 | | Claudetta Sanderson | 6 | 350-766-4559 | 2016-11-04 | 2018 | 8 | | Claudetta Sanderson | 6 | 350-766-4559 | 2016-11-04 | 2016 | 11 | | Bentley Oddie | 7 | 446-519-0975 | 2016-05-02 | 2018 | 8 | | Bentley Oddie | 7 | 446-519-0975 | 2016-05-02 | 2016 | 5 | | Theressa Dowker | 8 | 864-330-9976 | 2012-09-26 | 2018 | 8 | | Theressa Dowker | 8 | 864-330-9976 | 2012-09-26 | 2012 | 9 | | Jenica Belcham | 9 | 347-248-4379 | 2011-05-02 | 2011 | 5 | | Jenica Belcham | 9 | 347-248-4379 | 2011-05-02 | 2018 | 8 | 虽然简便些但却剥夺了sort的并行性，使他只能和distribute by 一样的列。但是可以实现输出文件的数据全排序