Mysql tuning note: INNER JOIN query Using temporary; Using filesort optimization problem

Recently I found a very slow implementation of sql in a production environment. Time is probably about 5s, and thus be on the scene to change the SQL EXPLAIN analysis, we found a pair appear during execution "Using temporary; Using filesort". That table is generated to store temporary values ​​during execution, ordered and sorted at the row pointer type, and all the lines connected to the memory sequencer and key matching conditions. After a reasonable optimization, after the final EXPLAIN, and no further details on SQL performance bottlenecks, and SQL execution time in less than 1s. Then has this experience been recorded.

Take a look at SQL and general table structure and data volume:

Orders Table Table order: data about 15w

CREATE TABLE `order` (
  `id` char(32) NOT NULL COMMENT 'id',
  `order_status` varchar(5) NOT NULL DEFAULT '0' COMMENT '订单状态',
  `order_time` datetime DEFAULT NULL COMMENT '下单时间',
   PRIMARY KEY (`id`)
)ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单数据';

Orders table associated with the payment schedule: data about 11w

CREATE TABLE `order_payment` (
  `id` char(32) NOT NULL COMMENT 'id',
  `order_id` char(32) NOT NULL DEFAULT '' COMMENT '订单id',
  `money` decimal(10,2) NOT NULL DEFAULT '0.00' COMMENT '付款金额',
 PRIMARY KEY (`id`),
 KEY `i_order_id` (`order_id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单付款数据';

After understanding about the structure of the table, look at a very common scene sql query, the query all waybills unfinished state orders and get their order ID and payment amount, and press the single reverse chronological order.

SQL is simple:

select o.id, op.money from `order` o 
inner join order_payment op on o.id = op.order_id 
where o.order_status = '4' order by o.order_time DESC;

By then EXPLAIN analysis, we found that:

select_type	table	type	possible_key	key	rows	Extra
SIMPLE	on	ALL	i_order_id	NULL	110000	Using temporary; Using filesort
SIMPLE	O	eq_ref	PRIMARY	PRIMARY	150000	Using where

For each column of the meanings herein above EXPLAIN is not much to say, are interested in can be found in the author's other Bowen has introduced.

First, learn Using temporary; Using filesort in the end what is, why it will be a performance killer.

Using temporary

Said that as sort does not take the index, the use of union, a subquery join queries, view the use of certain other reasons, thus creating a temporary internal table. Note that the temporary table may be a temporary table on memory, there may be a temporary table on the hard disk, of course, consumption is certainly better than the temporary table based on the actual consumption of a small hard disk time-based temporary tables memory.
When mysql creating a temporary table, select the memory or the hard disk temporary table temporary table depends on the maximum capacity parameter and max_heap_table_size tmp_table_size, memory temporary tables and minimum value tmp_table_size max_heap_table_size value, when the capacity of the temporary table is greater than the desired minimum of both value, mysql will use the hard drive to store temporary data tables.

Whether it is based on memory, or hard disk based. The process of building a temporary table is a time-consuming operation

Using filesort

Using filesort meaning of the word can not look the business, it does not mean that the use of file sorting. Only that there is no sort of use the index.
filesort the QuickSort algorithm is used, i.e., sorting of record metadata required to generate sorted block, and then use the method mergesort binning block. Wherein the memory space can be used filesort size parameter sort_buffer_size default is 2M. When ordering too many records sort_buffer_size not enough time, mysql will use temporary files to store various sub-blocks, each block and then merge sort and then repeatedly block the final completion of the global order.
My personal understanding is that, Using filesort not necessarily a performance killer. Using filesort because the only way to optimize away is to let go Sort index field, but this case should be a choice based on the actual situation, after all, there is a price index.

Understand the reason for the above two issues arising after, sql and is to appear before looking back at these two issues.

Using filesort Needless to say, it did not take the index caused. Key point of view Using temporary. Why would produce a temporary table.

MYSQL optimizer: in order to select JOIN

When Mysql inner join join statement encountered, MySQL algorithm table associated Nest Loop Join (nested loop join), Nest Loop Join matching operation is performed by two sequential cycle means, and returns the result set. SQL statements only describe objects and rules want to connect to, and implementation plan and execute operations to be realistic recorded line by line to match. Nest Loop Join operation is very simple, much like our simplest sort search algorithm, two-cycle structure. Two sets of data (data table) for connection are referred to as the outer table (table driven) and the inner table (non-driving table). Mysql to determine what will happen, which is the driving table tables, which tables Africa and Africa is driving table it? It mysql minimum data table as a table-driven table (i.e. base table), while the other table is called a non-driven table, the table is processed first drive qualified data for each row, and each row of data after non-drive connection table matching operation, until the end of the cycle, the final merge result, returns the results to the user. For field drive table it can be directly ordered, but the sort field for sorting tables need to merge the non-drive cycle results query (temporary tables), and therefore, when the order by o.order_time, on the first produced using temporary ( use temporary tables).

Earlier we know the amount of data order_payment only 11w, so of course order_payment is the driving table. So, to avoid using temporary, it is necessary to use the order as the driving table, this time STRAIGHT_JOIN keyword came.

STRAIGHT_JOIN strong designated driver table

explain select o.id, op.money from `order` o straight_join order_payment op on o.id = op.order_id  where o.order_status = '4' order by o.order_time DESC;

The results you will find Using temporary has disappeared.

Using filesort out the best solution is very simple, just give order_time index increased just fine.

But note that in this scenario the query, not just a field of order_time index increased, so is unavoidable Using filesort of. Because the characteristics of innerDB index, stored in the leaf node B TREE tree is the primary key index and column data ID. When only one order_time index, the query because there order_status in, you can not use the index

Best indexing scheme

create index i_order_status_time on `order`(`order_status`,`order_time`)

Such B TREE tree leaf node is stored in the index column data will contain (and order_status The order_time) and the primary key ID. The index can go both ordering inquiries. Achieve the best performance in the conditions described above queries.

SQL final analysis:

select_type	table	type	possible_key	key	rows	Extra
SIMPLE	O	ref	PRIMARY,i_order_status_time	i_order_status_time	150000	Using where; Using index
SIMPLE	on	ref	i_order_id	i_order_id	110000

Significantly improve query time.