A female colleague asked me how to query MySQL recursively? I acted as fiercely as a tiger, and watched her beating

Preface

The business scenario I'm doing recently involves recursive queries of the database. The Oracle used by our company, as we all know, Oracle has its own recursive query function, so it is very simple to implement.

However, I remember that MySQL does not have a recursive query function. How should it be implemented in MySQL?

So, there is this article.

The main knowledge points of the article:

• Oracle recursive query, start with connect by prior usage
• find_in_set function
• concat，concat_ws，group_concat 函数
• MySQL custom function
• Manually implement MySQL recursive query

Oracle recursive query

In Oracle, the recursive query is implemented through start with connect by prior syntax.

According to whether the prior keyword is at the byte point end or the parent node end, and whether it contains the current query node, there are four cases.

prior at the child node side (recursive downwards)

The first case: start with child node id ='query node' connect by prior child node id = parent node id

select * from dept start with id='1001' connet by prior id=pid;

Here, according to the condition id='1001', the current node and its child nodes are recursively searched. The query result includes itself and all child nodes.

 

The second case: start with parent node id ='query node' connect by prior child node id = parent node id

select * from dept start with pid='1001' connect by prior id=pid;

Here, according to the condition pid='1001', all child nodes of the current node are queried recursively. The query result only contains all its child nodes, not itself .

 

In fact, it is right to think about it, because the start condition is based on the parent node as the root node, and recursively downwards, naturally not including the current node.

prior on the parent node side (upward recursion)

The third case: start with child node id ='query node' connect by prior parent node id = child node id

select * from dept start with id='1001' connect by prior pid=id;

Here, according to the condition id='1001', the current node and its parent node are recursively searched. The query result includes itself and all its parent nodes.

 

The fourth case: start with parent node id ='query node' connect by prior parent node id = child node id

select * from dept start with pid='1001' connect by prior pid=id;

Here, according to the condition pid='1001', the first generation child node of the current node and its parent node are recursively queried. The query result includes its own first-generation child nodes and all parent nodes. ( Including myself )

 

In fact, this situation is easy to understand, because the query start condition is based on the parent node as the root node and recursively upwards. Naturally, it is necessary to include the first-level child nodes of the current parent node.

The above four situations may be confusing at first glance and easy to remember, but they are not.

We only need to remember that  the position of the priority is on the child node, and then recurse downward, and recurse upward on the parent node.

• If the start condition is a child node, it naturally includes its own node.
• If the start condition is the parent node, the current node is naturally not included when recursing downward. The upward recursion needs to include the current node and its first generation child nodes.

MySQL recursive query

As you can see, Oracle is very convenient to implement recursive queries. However, it does not help us in MySQL, so we need to manually implement the recursive query ourselves.

For convenience, we create a department table and insert several pieces of data that can form a recursive relationship.

DROP TABLE IF EXISTS `dept`;
CREATE TABLE `dept`  (
  `id` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  `pid` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1000', '总公司', NULL);
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1001', '北京分公司', '1000');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1002', '上海分公司', '1000');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1003', '北京研发部', '1001');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1004', '北京财务部', '1001');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1005', '北京市场部', '1001');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1006', '北京研发一部', '1003');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1007', '北京研发二部', '1003');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1008', '北京研发一部一小组', '1006');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1009', '北京研发一部二小组', '1006');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1010', '北京研发二部一小组', '1007');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1011', '北京研发二部二小组', '1007');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1012', '北京市场一部', '1005');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1013', '上海研发部', '1002');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1014', '上海研发一部', '1013');
INSERT INTO `dept`(`id`, `name`, `pid`) VALUES ('1015', '上海研发二部', '1013');

Yes, just now Oracle recursively used this table.

 

In addition, before this, we need to review a few functions in MYSQL, which will be used later.

find_in_set function

Function syntax: find_in_set(str,strlist)

str represents the string to be queried, strlist is a string separated by commas, such as ('a,b,c').

This function is used to find the position of the string str in the string strlist, and the return result is 1 ~ n. If it is not found, 0 is returned.

for example:

select FIND_IN_SET('b','a,b,c,d'); 

The result returns 2. Because the position of b is the second substring position.

 

In addition, it has another usage when querying table data, as follows:

select * from dept where FIND_IN_SET(id,'1000,1001,1002'); 

The result returns all the records with id in strlist, that is, id = '1000', id = '1001', id = '1002' three records.

 

Seeing this, I don’t know if you have any inspiration for the recursive query we are trying to solve.

Take the downward recursive query of all child nodes as an example. I wonder if it is possible to find a comma-concatenated string strlist containing the current node and all child nodes and pass it to the find_in_set function. You can query all the required recursive data.

So, the question now is how to construct such a string strlist.

This requires the following string splicing functions.

concat，concat_ws，group_concat 函数

1. In the string splicing function, the most basic is concat. It is used to concatenate N strings, such as,

select CONCAT('M','Y','S','Q','L') from dual; 

The result is a string of'MYSQL'.

 

Second, concat uses comma as the default separator, and concat_ws can specify the separator. The first parameter is passed in the separator, such as underscore.

 

3. The group_concat function is more powerful. It can concatenate fields into strings with specific separators while grouping.

Usage: group_concat ([distinct] the field to be connected [order by sort field asc/desc ][separator'separator'])

You can see that there are optional parameters, you can de-duplicate the field values ​​to be spliced, you can also sort, and specify the separator. If not specified, they are separated by commas by default.

For the dept table, we can concatenate all the ids in the table with commas. (The group by grouping field is not used here, so it can be considered that there is only one group)

 

MySQL custom function to realize recursive query

It can be found that the above problem of string splicing has also been solved. Then, the question becomes how to construct a recursive string.

We can customize a function and find all its child nodes by passing in the root node id.

Take downward recursion as an example.  (While explaining how to write custom functions, explain recursive logic)

delimiter $$ 
drop function if exists get_child_list$$ 
create function get_child_list(in_id varchar(10)) returns varchar(1000) 
begin 
    declare ids varchar(1000) default ''; 
    declare tempids varchar(1000); 
     set tempids = in_id; 
    while tempids is not null do 
        set ids = CONCAT_WS(',',ids,tempids); 
        select GROUP_CONCAT(id) into tempids from dept where FIND_IN_SET(pid,tempids)>0;  
    end while; 
    return ids; end  
$$ delimiter ; 

(1) delimiter $$, used to define the terminator. We know that MySQL's default terminator is a semicolon, which indicates that the instruction is over and executed. But in function problems, sometimes we hope that the semicolon does not end, so we need to temporarily change the terminator to a random other value. I set it here as $$, which means it ends when it encounters $$ and executes the current statement.

(2) drop function if exists get_child_list$$. If the function get_child_list already exists, delete it first. Note that you need to use the current custom ending character $$ to end and execute the statement. Because it needs to be distinguished separately from the function body below.

(3) create function get_child_list Create function. And the parameter is passed in the id of the child node of a root node, you need to pay attention to the type and length of the parameter, such as varchar(10) here. returns varchar(1000) is used to define the return value parameter type.

(4) The function body is surrounded by begin and end. Used to write specific logic.

(5) declare is used to declare variables, and default values ​​can be set with default.

The ids defined here is used as the return value of the entire function, which is used to concatenate into the recursive string separated by commas.

And tempids is to record the string of commas concatenated by all the child nodes temporarily generated in the while loop below.

(6) set is used to assign values ​​to variables. Here assign the passed root node to tempids.

(7) while do ... end while; loop statement, including loop logic. Note that you need to add a semicolon at the end of end while.

In the loop body, first use the CONCAT_WS function to join the final result ids and the temporarily generated tempids with a comma.

Then use FIND_IN_SET(pid,tempids)>0 as the condition, traverse all pids in tempids, find all child node ids with this as the parent node, and use GROUP_CONCAT(id) into tempids to splice these child node ids with commas Get up and overwrite update tempids.

When the loop comes in next time, ids will be spliced ​​again, and all child nodes of all child nodes will be searched again. Iterates back and forth, recursively traversing the child nodes layer by layer. Until it is judged that tempids is empty, indicating that all child nodes have been traversed, the whole loop is ended.

Here, use '1000' as an example, which is: (refer to the table data relationship in Figure 1)

第一次循环：
  tempids=1000  ids=1000    tempids=1001,1002 (1000的所有子节点)
第二次循环：  tempids=1001,1002  ids=1000,1001,1002  tempids=1003,1004,1005,1013 (1001和1002的所有子节点)
第三次循环：  tempids=1003,1004,1005,1013 
  ids=1000,1001,1002,1003,1004,1005,1013 
  tempids=1003和1004和1005及1013的所有子节点
...最后一次循环，因找不到子节点，tempids=null，就结束循环。

(8) return ids; Used to return ids as the function return value.

(9) After the function body is over, remember to use the terminator $$ to end the whole logic and execute it.

(10) Finally, don't forget to reset the terminator to the default terminator semicolon.

After the custom function is done, we can use it to recursively query the data we need. For example, I query all child nodes of the Beijing R&D department.

 

The above is a downward recursive query of all child nodes, and the current node is included. You can also modify the logic to not include the current node. I will not demonstrate it.

Manually implement recursive query (upward recursion)

Relative to downward recursion, upward recursion is relatively simple.

Because when recursing downwards, each level of recursion has a parent node corresponding to multiple child nodes.

In the upward recursion, each recursive child node corresponds to only one parent node, and the relationship is relatively simple.

Similarly, we can define a function get_parent_list to get all the parent nodes of the root node.

delimiter $$ 
drop function if exists get_parent_list$$ 
create function get_parent_list(in_id varchar(10)) returns varchar(1000) 
begin 
    declare ids varchar(1000); 
    declare tempid varchar(10); 
         set tempid = in_id; 
    while tempid is not null do 
        set ids = CONCAT_WS(',',ids,tempid); 
        select pid into tempid from dept where id=tempid; 
    end while; 
    return ids; end 
$$ delimiter ;  

Find the first team of Beijing R&D Two, and its recursive parent node, as follows:

 

Precautions

We use the group_concat function to concatenate strings. However, it should be noted that it has a length limit, the default is 1024 bytes. It can be viewed by show variables like "group_concat_max_len";.

Note that the unit is bytes, not characters. In MySQL, a single letter occupies 1 byte, while under utf-8, which we usually use, a Chinese character occupies 3 bytes.

This is very fatal for recursive queries. Because in general recursion, the relationship level is relatively deep, and it is likely to exceed the maximum length. (Although generally spliced ​​are digital strings, that is, single-byte)

So, we have two ways to solve this problem:

1. Modify the MySQL configuration file my.cnf, increase group_concat_max_len = 102400 #The maximum length you want.
2. Execute any of the following statements. SET GLOBAL group_concat_max_len=102400; or SET SESSION group_concat_max_len=102400; The difference between them is that global is global, and any new session opened will take effect, but note that the current session that has been opened will not take effect. The session will only take effect in the current session, and other sessions will not take effect. The common point is that they will all become invalid after MySQL restarts, and the configuration in the configuration file shall prevail. Therefore, it is recommended to modify the configuration file directly. The length of 102400 is generally sufficient. Assuming that the length of an id is 10 bytes, it can also spell 10,000 id.

In addition, there is another limitation in using the group_concat function, that is, you cannot use limit at the same time. Such as,

 

I only wanted to check 5 pieces of data for splicing, but it doesn't take effect now.

However, if needed, it can be achieved through subqueries,

If you like the article, please like it, comment and forward it, follow the editor, and the follow-up editor will bring a richer learning content update, I hope everyone will like it.