ProxySQL官档翻译__20_Mirroring

数据库 2020-04-28 18:38:52 阅读次数: 0

20_Mirroring

备注：文章编写时间201904-201905期间，后续官方在github的更新没有被写入

~
~
镜像[Mirroring]
注意：
1）它包含的规则可以随时改变；
2）它不支持预备(prepare)语句；

一、mysql_query_rules的扩展[Extensions to mysql_query_rules]

修改了 mysql_query_rules 表，添加了2个列：
1） mirror_flagOUT
2） mirror_hostgroup

因此，mysql_query_rules 表的新的定义变为：

Admin> show create table mysql_query_rules\G
*************************** 1. row ***************************
       table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
    rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
    username VARCHAR,
    schemaname VARCHAR,
    flagIN INT CHECK (flagIN >= 0) NOT NULL DEFAULT 0,
    client_addr VARCHAR,
    proxy_addr VARCHAR,
    proxy_port INT CHECK (proxy_port >= 0 AND proxy_port <= 65535), digest VARCHAR,
    match_digest VARCHAR,
    match_pattern VARCHAR,
    negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
    re_modifiers VARCHAR DEFAULT 'CASELESS',
    flagOUT INT CHECK (flagOUT >= 0), 
    replace_pattern VARCHAR CHECK(CASE WHEN replace_pattern IS NULL THEN 1 WHEN replace_pattern IS NOT NULL AND match_pattern IS NOT NULL THEN 1 ELSE 0 END),
    destination_hostgroup INT DEFAULT NULL,
    cache_ttl INT CHECK(cache_ttl > 0),
    cache_empty_result INT CHECK (cache_empty_result IN (0,1)) DEFAULT NULL,
    cache_timeout INT CHECK(cache_timeout >= 0),
    reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
    timeout INT UNSIGNED CHECK (timeout >= 0),
    retries INT CHECK (retries>=0 AND retries <=1000),
    delay INT UNSIGNED CHECK (delay >=0),
    next_query_flagIN INT UNSIGNED,
    mirror_flagOUT INT UNSIGNED,
    mirror_hostgroup INT UNSIGNED,
    error_msg VARCHAR,
    OK_msg VARCHAR,
    sticky_conn INT CHECK (sticky_conn IN (0,1)),
    multiplex INT CHECK (multiplex IN (0,1,2)),
    gtid_from_hostgroup INT UNSIGNED,
    log INT CHECK (log IN (0,1)),
    apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
    comment VARCHAR)
1 row in set (0.00 sec)

二、功能实现概述[Implementation overview]

当为匹配查询的规则设置了 mirror_flagOUT 或 mirror_hostgroup 时，将自动启用镜像实时查询功能。

请注意：
如果在规则中设置了 mirror_flagOUT 或 mirror_hostgroup 时又设置了replace_pattern：也就是如果原始查询的文本内容(SQL语句)已被重写(即查询规则设置了replace_pattern对原始SQL进行重写/替换)，则会为最终执行的查询(被修改后的语句)启用镜像查询功能，而会为原始SQL启用镜像查询：即如果查询(SQL语句)在根据digest、match_digest或match_pattern匹配到了查询规则，却发现该查询规则设置了replace_pattern对匹配到的查询SQL进行重写，那么，这时镜像逻辑将应用于被重写后的查询(SQL)。虽然被镜像的查询(即原始SQL语句)可以被重新编写或修改，但只要设置了mirror_flagOUT 或 mirror_hostgroup就会启用镜像查询功能。详情在后面。

如果源查询(SQL)与多个查询规则匹配，则可能会多次更改 mirror_flagOUT 或 mirror_hostgroup 。

镜像逻辑如下：
1）如果在处理源查询时设置了 mirror_flagOUT 或 mirror_hostgroup (查询规则设置的)，则会创建一个新的mysql会话。
2）新的mysql会话将获得原始mysql会话的所有相同属性：相同的凭据，库名，默认主机组等（注意：charset当前未被复制）
3）如果在原始会话中设置了 mirror_hostgroup ，则新会话将其默认主机组更改为 mirror_hostgroup 。
4）如果未设置 mirror_flagOUT ，则新会话将针对定义的 mirror_hostgroup 执行原始查询。
5）如果在原始会话中设置了 mirror_flagOUT ，则新的mysql会话将尝试根据原始会话的mirror_flagOUT的值在 mysql_query_rules 中查找FlagIN值与之相等的查询规则(即查找FlagIN=mirror_flagOUT的规则)； 然后将镜像查询请求发送到这个规则中进行处理：这样就可以修改查询，如重写查询，或再次更改 hostgroup 。
(参考后面的<七、高级示例：使用镜像测试查询重写>内容)

三、案例一：原始查询与镜像查询选择同一主机组[Mirror selects to same hostgroup]

在这个非常简单的示例中，我们将把所有SELECT语句发送到hostgroup10，包括原始语句和镜像语句。

1、设置查询规则

1）设置查询规则

将原始查询与镜像查询指向同一主机组：

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
1 row in set (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (6,1,'^SELECT',10,10,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
| 6       | 1      | ^SELECT       | 10                    | 10               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
2 rows in set (0.00 sec)

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM runtime_mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
| 6       | 1      | ^SELECT       | 10                    | 10               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
2 rows in set (0.00 sec)

2、连接ProxySQL执行查询

从mysql会话中我们将运行一些查询：

1）使用业务账号连入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空并查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 1          | sbtest     | show databases                 |
| 10        | 1          | sbtest     | show tables                    |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.00 sec)

3）业务账号执行查询

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec)

4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 2          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
1 row in set (0.00 sec)

我们可以看到 SELECT 语句被执行了两次！！

5）作为附加测试，我们重新运行相同的查询：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec

6）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 4          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
1 row in set (0.01 sec)

count_star是我们执行查询次数的两倍，因为它是镜像的。值得注意的是，ProxySQL会收集原始查询和镜像查询的指标。

四、案例二：原始查询与镜像查询选择到不同的主机组[Mirror SELECT to different hostgroup]

在此示例中，我们将重新配置proxysql以将所有SELECT语句发送到hostgroup10，但要在hostgroup20上执行镜像查询：

1、设置查询规则

1）设置查询规则

Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (5,1,'^SELECT',10,20,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | ^SELECT       | 10                    | 20               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
1 row in set (0.00 sec)

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

2、连接ProxySQL执行查询

1）使用业务账号连入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以获得新的统计信息：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+----------------+
| hostgroup | count_star | schemaname | digest_text    |
+-----------+------------+------------+----------------+
| 10        | 1          | sbtest     | show databases |
| 10        | 1          | sbtest     | show tabels    |
| 10        | 2          | sbtest     | show tables    |
+-----------+------------+------------+----------------+
3 rows in set (0.00 sec)

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
Empty set (0.00 sec)

3）业务账号执行查询

从mysql客户端我们现在可以运行一些查询（为简单起见，我们运行相同）：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec)

4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 20        | 1          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
| 10        | 1          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.00 sec)

可以看到ProxySQL向hostgroup10和hostgroup20发送了相同的相同查询！

五、案例三：重写源查询和镜像[rewrite both source query and mirror]

在这个例子中，我们将重写原始查询，然后镜像它：为简单起见，我们将对表 sbtest[0-9]+ 的操作重写为对表 sbtest3 的操作：

1、设置查询规则

1）设置查询规则

Admin> DELETE FROM mysql_query_rules;
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply) VALUES (5,1,'sbtest[0-9]+',10,'sbtest3',20,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | replace_pattern | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
| 5       | 1      | sbtest[0-9]+  | 10                    | sbtest3         | 20               | 1     |
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
1 row in set (0.00 sec)

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

2、连接ProxySQL执行查询

1）使用业务账号连入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以获得新的统计信息：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+----------------+
| hostgroup | count_star | schemaname | digest_text    |
+-----------+------------+------------+----------------+
| 10        | 1          | sbtest     | show databases |
| 10        | 2          | sbtest     | show tables    |
+-----------+------------+------------+----------------+
2 rows in set (0.00 sec)

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
Empty set (0.01 sec)

3）业务账号执行查询

从mysql客户端我们可以运行通常的查询：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1149 |
| 9825 |
| 5704 |
+------+
3 rows in set (0.00 sec)

正如预期的那样，这次查询输出的结果与前一个不同，因为现在重写了原始查询。表名是是查询了sbtest1其实被改写到了sbtest3;通过下面内容可以验证。

4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 20        | 1          | sbtest     | SELECT id FROM sbtest3 LIMIT ? |
| 10        | 1          | sbtest     | SELECT id FROM sbtest3 LIMIT ? |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.01 sec)

正如所料，修改后的查询在两个主机组上执行，而且原始SQL中的表名sbtest1被重写为了sbtest3。

六、仅重写镜像查询[rewrite mirror query only]

在此示例中，我们将只重写镜像查询。
这非常有用，例如，我们想要了解重写查询的性能，或者新索引是否会提高性能。
在此示例中，我们将以使用和不使用索引来比较相同查询的性能。当然，我们也会将查询发送到相同的主机组。

1、设置查询规则

1）设置查询规则

创建以下规则（rule_id = 5）：
1）匹配 FROM sbtest1 ;
2）设置destination_hostgroup = 20 ;
3）设置mirror_flagOUT=100 ;
4）不设置mirror_hostgroup ;

Admin> DELETE FROM mysql_query_rules;
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'FROM sbtest1 ',20,100,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | FROM sbtest1  | 20                    | 100            | NULL             | 1     |
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
1 row in set (0.00 sec)

由于设置了 mirror_flagOUT ，因此将创建一个新会话来运行相同的查询。但是，因为未设置 mirror_hostgroup ，所以会根据当前用户在 mysql_users 中设置的默认主机组，将查询将被发送到其默认主机组上。相反的，如果我们希望将镜像查询发送到与原始主机组相同的主机组。我们可以在rule_id = 5的规则中设置mirror_hostgroup，或者创建
一个新规则。这里，选择后者，我们将创建一个新规则来匹配重写查询：

Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES (10,1,100,'FROM sbtest1 ',20,'FROM sbtest1 IGNORE INDEX(k_1) ',1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| rule_id | active | flagIN | match_pattern | replace_pattern                 | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | 0      | FROM sbtest1  | NULL                            | 20                    | 100            | NULL             | 1     |
| 10      | 1      | 100    | FROM sbtest1  | FROM sbtest1 IGNORE INDEX(k_1)  | 20                    | NULL           | NULL             | 1     |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
2 rows in set (0.00 sec)

或查看规则的全部信息：

Admin> SELECT * FROM mysql_query_rules \G 
*************************** 1. row ***************************
              rule_id: 5
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 0
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: NULL
        match_pattern: FROM sbtest1 
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: NULL
destination_hostgroup: 20
            cache_ttl: NULL
   cache_empty_result: NULL
        cache_timeout: NULL
            reconnect: NULL
              timeout: NULL
              retries: NULL
                delay: NULL
    next_query_flagIN: NULL
       mirror_flagOUT: 100
     mirror_hostgroup: NULL
            error_msg: NULL
               OK_msg: NULL
          sticky_conn: NULL
            multiplex: NULL
  gtid_from_hostgroup: NULL
                  log: NULL
                apply: 1
              comment: NULL
*************************** 2. row ***************************
              rule_id: 10
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 100
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: NULL
        match_pattern: FROM sbtest1 
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: FROM sbtest1 IGNORE INDEX(k_1) 
destination_hostgroup: 20
            cache_ttl: NULL
   cache_empty_result: NULL
        cache_timeout: NULL
            reconnect: NULL
              timeout: NULL
              retries: NULL
                delay: NULL
    next_query_flagIN: NULL
       mirror_flagOUT: NULL
     mirror_hostgroup: NULL
            error_msg: NULL
               OK_msg: NULL
          sticky_conn: NULL
            multiplex: NULL
  gtid_from_hostgroup: NULL
                  log: NULL
                apply: 1
              comment: NULL
2 rows in set (0.00 sec)

需要注意的是，在rule_id = 10的规则中，镜像查询将匹配该规则[因为mirror_flagOUT(5号规则)=flagIN(100=10号规则)]，此时我们需要设置 destination_hostgroup 而不是 mirror_hostgroup ：只应该为原始查询设置mirror_hostgroup ，以便立即可以将镜像查询请求发送到指定位置，而无需 mysql_query_rules 中的其他额外规则介入。

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM runtime_mysql_query_rules ;
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| rule_id | active | flagIN | match_pattern | replace_pattern                 | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | 0      | FROM sbtest1  | NULL                            | 20                    | 100            | NULL             | 1     |
| 10      | 1      | 100    | FROM sbtest1  | FROM sbtest1 IGNORE INDEX(k_1)  | 20                    | NULL           | NULL             | 1     |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
2 rows in set (0.00 sec)

配置已生效！

2、业务账号执行查询

1）使用业务账号连入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以获得新的统计信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 0        |
+----------+
1 row in set (0.00 sec)

3）业务账号执行查询

从mysql客户端我们可以运行通常的查询：

mysql> SELECT id FROM sbtest1 ORDER BY k DESC LIMIT 3;
+------+
| id   |
+------+
| 5076 |
| 5121 |
| 8573 |
+------+
3 rows in set (0.00 sec)

mysql> SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT 3;
+------+------+
| id   | k    |
+------+------+
| 5076 | 7672 |
| 5121 | 7655 |
| 8573 | 7565 |
+------+------+
3 rows in set (0.01 sec)

4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> SELECT hostgroup,count_star,sum_time,digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC;
+-----------+------------+----------+--------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                        |
+-----------+------------+----------+--------------------------------------------------------------------+
| 20        | 1          | 3997     | SELECT id,k FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? |
| 20        | 1          | 3997     | SELECT id FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ?   |
| 20        | 1          | 615      | SELECT id FROM sbtest1 ORDER BY k DESC LIMIT ?                     |
| 20        | 1          | 342      | SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT ?                   |
+-----------+------------+----------+--------------------------------------------------------------------+
4 rows in set (0.01 sec)

表stats_mysql_query_digest 的结果表明：
1）原始查询被镜像了；
2）原始查询未被重写(而是镜像)；
3）镜像查询被重写了(带了规则里的IGNORE语句)；
4）镜像查询速度要慢得多，因为忽略了索引；

七、高级示例：使用镜像测试查询重写[Advanced example: use mirroring to test query rewrite]

在处理镜像时，我被问到一个完全不同的问题，与查询重写有关：如何知道给定的正则表达式是否与给定的查询匹配，并验证重写模式是否正确？
更具体地说，问题是要了解重写是否正确而不影响实时流量。虽然镜像最初并不是为此设计的，但它可以回答这个问题。

在这个例子中，我们将编写一个规则来匹配所有SELECT，"镜像"它们，并尝试重写它们。

1、设置查询规则

1）设置查询规则

Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'^SELECT ',20,100,1);
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES 
(10,1,100,'^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$',20,'SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c',1);
Query OK, 1 row affected (0.00 sec)

查看设置结果：

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM mysql_query_rules \G
*************************** 1. row ***************************
              rule_id: 5
               active: 1
               flagIN: 0
        match_pattern: ^SELECT 
destination_hostgroup: 20
      replace_pattern: NULL
       mirror_flagOUT: 100
                apply: 1
*************************** 2. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
2 rows in set (0.00 sec)

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME ;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules \G
*************************** 1. row ***************************
              rule_id: 5
               active: 1
               flagIN: 0
        match_pattern: ^SELECT 
destination_hostgroup: 20
      replace_pattern: NULL
       mirror_flagOUT: 100
                apply: 1
*************************** 2. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
2 rows in set (0.00 sec)

上面的正则表达式非常复杂，这就是为什么镜像查询功能有用之处，它不是直接重写实时流量。

2、业务账号执行查询

1）使用业务账号连入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以获得新的统计信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 10       |
+----------+
1 row in set (0.00 sec)

3）业务账号执行查询

从mysql客户端我们可以运行通常的查询：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

查询已成功运行。 如上所述，我们没有修改原始流量。
那 stats_mysql_query_digest 中是又怎么样？

4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 20        | 2          | 1493     | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
1 row in set (0.00 sec)

可以看到，原始查询执行了两次，因此某些内容无法正常运行。我们可以注意到两个查询都被发送到hostgroup20：我们应该相信rule_id = 10是匹配的，但是没有重写查询。
让我们验证它是匹配的：

Admin> SELECT * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 5       | 2    |  -->其中有一次为show tables触发的。
| 10      | 1    |
+---------+------+
2 rows in set (0.00 sec)

根据 stats_mysql_query_rules 中的规则命中信息可以看到，rule_id = 10的规则是匹配的。
那为什么不重写查询？

5）问题修复

在重新审查一次rule_id=10的replace_pattern内容：

SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c

发现，+号前面不该有转义的。更新查询规则10：

Admin> UPDATE mysql_query_rules SET replace_pattern='SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c' WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
1 row in set (0.00 sec)

修改已经生效！！

6）验证结果

ProxySQL管理端清空stats_mysql_query_digest以获得新的统计信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 5        |
+----------+
1 row in set (0.00 sec)

在客户端再次执行原SQL：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

现在让我们验证查询是否被正确重写：

Admin>  select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 20        | 1          | 414      | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c           |
| 20        | 1          | 661      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)

到此可以看到，查询被正确重写，并且执行了！

八、高级示例：使用镜像和防火墙来测试查询重写[Advanced example: use mirroring and firewall to test query rewrite]

前面的示例/练习有点超前：那么，是否可以在不执行查询的情况下重写查询？答案是：可以的！
为此，我们将为镜像查询设置 error_msg ：这样ProxySQL将处理镜像查询，但会过滤它而不将其发送到任何mysql服务器。
如最初所讲的那样，可以修改镜像查询，并且防火墙是修改镜像查询的示例。

例如：
接着前面的案例，继续操作。

1）修改查询规则

为规则10添加error_msg信息。

Admin> UPDATE mysql_query_rules SET error_msg="random error, blah blah" WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)

2）加载配置到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.01 sec)

查看修改结果：

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply,error_msg FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
            error_msg: random error, blah blah
1 row in set (0.00 sec)

可以看到，修改已生效！！

3）ProxySQL管理端清空并查看当前以执行SQL的查询概要统计：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 0        |
+----------+
1 row in set (0.00 sec)

4）业务账号执行查询

在mysql客户端重新运行查询：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

5）ProxySQL管理端，查看当前以执行SQL的查询概要统计：

Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 10        | 1          | 0        | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c           |
| 20        | 1          | 730      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)

Admin> SELECT * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 5       | 1    |
| 10      | 1    |
+---------+------+
2 rows in set (0.01 sec)

Great!!我们已看到查询已被重写，但实际上并未在任何地方发送：
1）sum_time = 0 ，因为响应是立即的;
2）hostgroup=10 ，在规则10中，设置了destination_hostgroup=20，但这里却显示了业务账号默认的主机组10；如果业务账号未设置默认主机组，则显示为0；这种显示和设置不一致也表面没有发送到任何地方(不信可以打开默认主机组MySQL的general_log去验证).

完毕！

猜你喜欢

转载自blog.51cto.com/4709096/2491163
今日推荐
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
GCC 14.1 发布
面壁智能发布 Eurux-8x22B 开源大模型 —— 堪称「理科状元」
开源日报 | 谷歌扶持鸿蒙上位；开源Rabbit R1；Docker加持的安卓手机；微软的焦虑和野心；海尔电器把开放平台关了
中国码农的“35岁魔咒”
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Arc Browser for Windows 1.0 正式 GA
90后程序员开发视频搬运软件、不到一年获利超 700 万，结局很刑！
周排行
基本数据类型封装类比较 Java源码解读(一) 8种基本类型对应的封装类型
JS实现无缝滚动上
深入解析HashMap原理（基于JDK1.8）
mysql的连接池
关于.htc
linux下的ubuntu12.04图形界面
【数论】好推不好记的扩展欧几里德
设备树详解
cscope + tags 简单设置
xml学习
每日归档
更多
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)
2024-05-03(19)
2024-05-02(0)
2024-05-01(4)
2024-04-30(1)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022