mysql tuning sorting optimization, subquery optimization

mysql tuning

Sort optimization

1. Eliminate using fileshort:

   Scenario 1: select id from table order by col1

   Add an index to the sort field (col1) (the index itself is ordered)

   Scenario 2: select id form table order by col1,col2

   Wrong approach: add a single-column index to them

   The correct way: add a joint index to col1 and col2, and the order should be consistent with the order after sorting order by

   Scenario 3: select id from table order by col1 asc, col2 desc

   Create indexes directly, the default is ascending order, still using filesort

   ALERT TABLE ta ADD INDEX idx_col1_col2(col1,col2)

   The solution is: specify the ascending and descending order when creating the index (only supported in version 8)

   ALERT TABLE ta ADD INDEX idx_col1_col2(col1 asc,col2 desc)

   Can the following sql avoid using filesort(idx_col1_col2(col1 ,col2 ,col3))

   select id from table order by col2,col1,col3; 否

   select id form table order by col1,col3; 否

   Must meet the leftmost prefix principle

   1	a	x
   1	a	y
   1	b	x
   1	b	z
   2	a	y
   2	a	z
   2	c	x
   2	c	z

   select id from table where col1=100 order by col2

   It is best to add a joint index to col1 and col2. If the conditions do not allow, add an index to col1 first, and filter the data at most

   Can the following SQL avoid using filesort:

   select id from table where col1 in(100,102) order by col2; 否

   select id from table where col1>=100 and col1<102 order by col2; 否

   After filtering by the previous conditions, the sorting fields are all unordered, and col2 is ordered only when col1 is specifically equal to a certain value.

   Scenario 4: select * from table order by col1, id limit 10000;

   After adding an index to col1, it is not necessary to sort by index, because MySQL will consider the cost of returning to the table.

   Optimization method:

   • order by and select fields plus joint index
   • By manually returning to the table:select a.* from table a inner join(select id from table order by col1,id limit 10000 )b on a.id=b.id;

   MySQL sorts memory sorting and external disk-assisted sorting. When the sort buffer cannot fit at one time, it will write to the file to ensure the internal order of the file, and then use the merge algorithm to make all the data in order.

   sort by:

   • rowid sorting (two-way sorting): the key-value pair composed of the sorting field and rowid (primary key) is stored in the sort buffer because it requires two disk operations to retrieve the value and return the table, so it is called two-way sorting. The advantage is that it occupies a small sort buffer, and the disadvantage is that to return to the table

   例子：select col2 from table order by col1;

• Full field sorting: the sorted fields and the fields that need to be returned form key-value pairs and are stored in the sort buffer. The advantage is that there is no need to return to the table, and the disadvantage is that it takes up a lot of sort buffer and it is easier to use temporary files

• Packed full field sorting: improvement of all fields, compression of query fields, so as to store more records in the sort buffer

sort_buffer_size: The parameter explains the size of the buffer used to perform the sort

Set sort cache command:

View parameter values: show variable like 'sort_buffer_size';

Modify the value of the parameter: (session and global level)

session sort_buffer_size = 1023*1024;

set global sort_buffer_size 1024*1024;

Set permanent effective modification /etc/my.cnf

The innodb_sort_buffer_size setting is the size of the sort_buffer used when creating the innodb index, which has nothing to do with sorting.

max_length_for_sort_data: Parameters explain the parameters provided to users to control the sorting algorithm, version 8 abolished

The total length of the returned column > max_length_for_sort_data ? rowid sorting: full field/package full field sorting

View parameter value: show variables where variable_name ='max_length_for_sort_data'default 1024 bytes

Modify parameter values:set max_length_for_sort_data=1024;

Enable optimizer trace command

Check which sorting method is used for sorting sql: optimizer trace

• Inquire:show variables where variable_name='optimizer_trace';

• Turn on:set optimizer_trace = ‘enable=on’;

• Control how many results to display at most:set optimizer_trace_limit =5;

• The first optimizer_trace offset to display (default 1):set optimizer_trace_offset=1;

• execute sort sql

• Query tracking information:select *from INFORMATION_SCHEMA.OPTIMIZER_TRACE;

subquery optimization

case:

select * from orders where id in (select order_id from order_details where product_code =“xxx”)

Each row in the outer layer calls select_type(DEPENDENT SUBQUERY) for the subquery. After version 8, the subquery will be optimized. The result of the first subquery is saved as a temporary table, and subsequent access to the subquery is directly obtained through the temporary table. The subquery only needs to be executed once in the whole process.

Optimize sql:

select t1.* from orders t1 inner join order_details t2 on t1.id=t2.order_id where t2.prodict_code=‘xxx’

Add index:

Joint index ADD INDEX idx_pcode_oid(product_code,order_id)

Summarize:

• Use table joins instead of subqueries, especially if the result set of the subquery is large
• Add conforming index and retrograde optimization, where the fields include were condition fields and associated fields
• The order of the fields in the index must follow the leftmost prefix principle
• mysql8 automatically optimizes subqueries, and the performance is close to that of table joins
• Try to make small tables drive large tables