How does Mysql perform efficient bulk insert and update library ON DUPLICATE KEY UPDATE

1. Requirements and solutions:

Requirements: Batch insertion, update if data exists, insert if data does not exist
Example:

 <insert id="batchUpdate" parameterType="java.util.List">
        insert into tb_list
        (col1,col2,col3,col4)
        values
        <foreach collection="list" item="item" index="index" separator=",">
            (#{item.col1},
            #{item.col2},
            #{item.col3},
            #{item.col4})
        </foreach>
        ON DUPLICATE KEY UPDATE
        col1= values(col1)
    </insert>

Effect: Update 1983 pieces of data within 2 seconds, including data comparison

2020-12-13 01:42:58.654 [http-nio-8100-exec-2] INFO  com...
2020-12-13 01:43:01.848 [http-nio-8100-exec-2] DEBUG c.m.d.d.u.TbQrcQrcWhiteListMapper.batchUpdate - ==>  Preparing: insert into ...
2020-12-13 01:43:02.553 [http-nio-8100-exec-2] DEBUG c.m.d.d.u.TbQrcQrcWhiteListMapper.batchUpdate - <==    Updates: 1983

2. The difference between replace into and insert into

replace into is similar to insert, the difference is: replace into first tries to insert data into the table,

1. If it is found that this row of data already exists in the table (judged by the primary key or unique index), delete this row of data first, and then insert new data.

2. Otherwise, insert new data directly.

It should be noted that the table where data is inserted must have a primary key or a unique index! Otherwise, replace into will insert data directly, which will cause duplicate data in the table

 

3.ON DUPLICATE KEY UPDATE

1: ON DUPLICATE KEY UPDATE needs to have a column with a primary key or a unique index in the INSERT statement, and the corresponding data is already in the table before performing the update operation. And if the field to be updated is a primary key or a unique index, it cannot be duplicated with the existing data in the table, otherwise both insert and update will fail.

2: Whether it is an update or an increase statement, it is not allowed to change the data of the corresponding field of the primary key or unique index into the data that already exists in the table