Are you still doing reading and writing separate code it, try this middleware it!

SpringBoot actual electricity supplier item mall (25k + star) Address: github.com/macrozheng/...

Summary

Traditional MySql read and write separation scheme is achieved by dynamically switching data source code according to the type of SQL statements, then there is no middleware can automatically separate read and write it? Millet open source database middleware Gaea can be achieved, then we will explain in detail how to read and write Gaea separation of MySql.

Gaea简介

Gaea is based on MySql database middleware protocol millet electricity supplier in China R & D research and development, currently widely used in millet Mall mainland and overseas, including more business orders, community activities. Basic features Gaea sub-library points table, SQL routing, separate read and write, etc., wherein the sub-library sub-table routing scheme compatible mycat and kingshard two items.

MySql master-slave replication

Gaea need to rely on using MySql master-slave replication environment, from the main MySql on copying can refer to: MySql master-slave replication, from principle to practice!

Installed directly under Linux

Currently provided by the official Linux is installed directly in the way, let this method to install Gaea.

Installation Go locale

Since Gaea is to use the Go language, so we need to install environmental Go language.

• Installation Go locale, Download: golang.org/dl/

• After the download is complete extract to /mydataa directory;

tar -zxvf go1.13.5.linux-amd64.tar.gz -C /mydata/
复制代码

• Add /mydata/go/bindirectory to the PATH variable:

# 编辑环境变量配置文件
vim /etc/profile
# 在最后一行添加
export GOROOT=mydata/go
export PATH=$PATH:$GOROOT/bin
# 刷新配置文件
source /etc/profile
复制代码

• View the version number, to test whether the installation was successful:

go version
复制代码

• Go returns the following information represents the locale has been installed successfully:

go version go1.13.5 linux/amd64
复制代码

安装Gaea

Since Gaea does not provide installation package, so we need to get to compile the source code executable file.

• Gaea download the source code, download zippackage to download address: github.com/XiaoMi/Gaea

• The download package good compression decompression operation, where we extract it to /mydata/gaea/a directory:

unzip Gaea-master.zip
复制代码

• Enter the /mydata/gaea/directory, use the makecommand to compile the source code:

make build
复制代码

• 注意: Due to network problems, some dependent Go for it will download the compilation fails, try several times to succeed;

• After compilation in /mydata/gaea/binthe directory will generate Gaea executable file gaea:

• Since we do not build a etcddistribution center, so it is necessary to modify the local configuration file /mydata/gaea/etc/gaea.ini, change the configuration type file:

; 配置类型，目前支持file/etcd两种方式，file方式不支持热加载
config_type=file
复制代码

• Add namespace configuration files for configuring database information from our Lord, profile Address:/mydata/gaea/etc/file/namespace/mall_namespace_1.json

• Configuration file as follows:

{
    "name": "mall_namespace_1",
    "online": true,
    "read_only": false,
    "allowed_dbs": {
        "mall": true
    },
    "slow_sql_time": "1000",
    "black_sql": [
        ""
    ],
    "allowed_ip": null,
    "slices": [
        {
            "name": "slice-0",
            "user_name": "root",
            "password": "root",
            "master": "192.168.6.132:3307",
            "slaves": ["192.168.6.132:3308"],
            "statistic_slaves": null,
            "capacity": 12,
            "max_capacity": 24,
            "idle_timeout": 60
        }
    ],
    "shard_rules": null,
    "users": [
        {
            "user_name": "macro",
            "password": "123456",
            "namespace": "mall_namespace_1",
            "rw_flag": 2,
            "rw_split": 1,
            "other_property": 0
        }
    ],
    "default_slice": "slice-0",
    "global_sequences": null
}
复制代码

namespace Profile

namespace configuration JSON format, sub-table contains configuration information, presumptuous table, examples, etc., can be changed at runtime.

• Overall configuration instructions:

  Field Name	Field Type	Field Meaning
  name	string	namespace name
  online	bool	Whether online, offline use logic
  read_only	bool	Whether read-only, namespace level
  allowed_dbs	map	Allow access to the database through a proxy
  default_phy_dbs	map	The default database name, and one-allowed_dbs
  slow_sql_time	string	Sql slow time in ms
  black_sql	string array	Sql blacklist
  allowed_ip	string array	IP whitelist
  slices	map array	Examples of a master from multiple physical, in particular in the field map can be referred to slice slice configuration
  shard_rules	map array	Configuration database is divided, sub-table, a special table, with reference to the specific field may be configured shard
  users	map array	Application of a user terminal connected gaea desired configuration, users can refer to the specific field configuration

• slice configuration:

  Field Name	Field Type	Field Meaning
  name	string	Fragment name, automatically generate an orderly
  user_name	string	database username
  password	string	Database Password
  master	string	Examples of primary address
  slaves	string array	Address from the database, can configure multiple
  statistic_slaves	string array	Statistics from the instance type Address List
  capacity	int	Connection pool size in each instance and gaea_proxy
  max_capacity	int	Each instance of the connection pool gaea_proxy maximum size
  idle_timeout	int	gaea_proxy connection with the backend mysql free survival time, unit: seconds

• users configuration:

  Field Name	Field Type	Field Meaning
  user_name	string	Database proxy user name, client name accessed by the user
  password	string	Acting password database, client access by the user name
  namespace	string	Corresponding namespace
  rw_flag	int	Identifying write, read-only = 1, read = 2
  rw_split	int	If separate read and write, read and write non-separation = 0, separate read and write 1 =
  other_property	int	Currently used to identify whether or not to go to statistics from the example, the average user = 0, 1 = user statistics

Run Docker containers

As the only official way to install direct running under Linux, here we provide an alternative mode of operation, in Docker container as a service.

Mirror packaged into Docker

Docker Hub and no packaged Gaea mirror, we need to build a self, the following details how to build Gaea under the Docker image.

• Here we use Dockerfile build Docker mirror, Dockerfile contents are as follows:

# 该镜像需要依赖的基础镜像
FROM golang:latest
# 将当前目录下的gaea源码包复制到docker容器的/go/Gaea-master目录下，对于.tar.gz文件会自动解压
ADD Gaea-master.tar.gz /go/Gaea-master
# 将解压后的源码移动到/go/gaea目录中去
RUN bash -c 'mv /go/Gaea-master/Gaea-master /go/gaea'
# 进入/go/gaea目录
WORKDIR /go/gaea
# 将gaea源码进行打包编译
RUN bash -c 'make build'
# 声明服务运行在13306端口
EXPOSE 13306
# 指定docker容器启动时执行的命令
ENTRYPOINT ["/go/gaea/bin/gaea"]
# 指定维护者的名字
MAINTAINER macrozheng
复制代码

• Prior to this we need to Gaea source compressed packets into .tar.gzconvenient format decompression in Docker containers can be used 压缩软件to achieve:

• After using the command building Docker Docker image of Gaea:

docker build -t gaea:1.0.2 .
复制代码

• Building a successful console output:

• Copy the local installation of Gaea configuration file to the /mydata/gaea-docker/etc/directory:

cp -r /mydata/gaea/etc/ /mydata/gaea-docker/etc/
复制代码

• Docker containers using the command to start Gaea:

docker run -p 13306:13306 --name gaea \
-v /mydata/gaea-docker/etc:/go/gaea/etc \
-d gaea:1.0.2
复制代码

Test separate read and write

Test ideas: First, we closed the copy from the primary instance, and then to operate the database by Gaea proxy, insert a piece of data, if there are primary examples of this data and not from an instance, the description of the write operation is to take the main library. And then query the table by Gaea proxy data, without this data, a read command to go from the library to read and write separation proved successful.

• Navicat to connect through a proxy Gaea, pay attention to here is the content namespace Gaea account password in the configuration, port Gaea service port;

• Navicat are connected by the main library from the library, and for viewing data, this time to establish the following three database connections;

• By stop slavecommand closes mysql-slaveexample replication from the master:

• Gaea agent by testinserting a data table:

• Check the main library in testthe data table, the data have been found:

• In the view from the library testtable data, we found no data to prove that the write operation is taking the main library:

• View agent directly in testthe data table, which found no data to prove a read operation is taking from the library.

Use in conjunction SpringBoot

SpringBoot in our application, we just need to Gaea proxy services directly as a database service to use a separate read and write can be achieved. Thus without adding any separate read and write logic in the code, it is not very convenient!

Reference material

For more information, please refer to the official document: github.com/XiaoMi/Gaea

the public

mall project a full tutorial serialized in public concern number the first time to obtain.