Spring's Road to God Chapter 52: Spring realizes database read-write separation
1. Background
Most systems read more and write less. In order to reduce the pressure on the database, multiple slave libraries can be created for the main library. The slave library automatically synchronizes data from the main library. The program sends the write operation to the main library, and the read operation Send it to the slave library for execution.
Today's main goal: to achieve read-write separation through spring .
Read-write separation needs to implement the following two functions:
1. The method of reading is controlled by the caller whether to read from the library or the main library
2. There is a transaction method, and all internal read and write operations go to the main library
2. Think about 3 questions
1. The method of reading is controlled by the caller, whether to read from the slave library or the main library. How to realize it?
You can add a parameter to all read methods to control whether to read from the library or the master library.
2. How to route the data source?
The spring-jdbc package provides an abstract class: AbstractRoutingDataSource, which implements the javax.sql.DataSource interface. We use this class as the data source class. The point is that this class can be used for data source routing and can be configured internally. There are multiple real data sources, and it is up to the developer to decide which data source to use in the end.
There is a map in AbstractRoutingDataSource to store multiple target data sources
private Map<Object, DataSource> resolvedDataSources;
For example, the master-slave library can be stored like this
resolvedDataSources.put("master",主库数据源);
resolvedDataSources.put("salave",从库数据源);
There is also an abstract method in AbstractRoutingDataSource determineCurrentLookupKey
, use the return value of this method as a key to find the corresponding data source in the above resolvedDataSources, as the data source for the current operation db
protected abstract Object determineCurrentLookupKey();
3. Where is the separation of reading and writing controlled?
Read-write separation is a general function, which can be realized by spring aop . Add an interceptor, before intercepting the target method, before the target method is executed, get which library you need to go currently, and store this flag in ThreadLocal. Use this flag as the return value of the AbstractRoutingDataSource.determineCurrentLookupKey() method. After the target method is executed in the interceptor, clear this flag from ThreadLocal.
3. Code implementation
3.1. Project structure diagram
[External link picture transfer failed, the source site may have an anti-theft link mechanism, it is recommended to save the picture and upload it directly (img-xzUJnT7h-1684759744195)(%E6%96%B0%E5%BB%BA%E6%96%87% E6%9C%AC%E6%96%87%E6%A1%A3/1369022-20211107221309265-2099055565.png)]
3.2、DsType
Indicates the type of data source, and has 2 values, which are used to distinguish whether it is the master library or the slave library.
package com.javacode2018.readwritesplit.base;
public enum DsType {
MASTER, SLAVE;
}
3.3、DsTypeHolder
There is a ThreadLocal inside, which is used to record whether the main library or the slave library is currently used, and this flag is placed in dsTypeThreadLocal
package com.javacode2018.readwritesplit.base;
public class DsTypeHolder {
private static ThreadLocal<DsType> dsTypeThreadLocal = new ThreadLocal<>();
public static void master() {
dsTypeThreadLocal.set(DsType.MASTER);
}
public static void slave() {
dsTypeThreadLocal.set(DsType.SLAVE);
}
public static DsType getDsType() {
return dsTypeThreadLocal.get();
}
public static void clearDsType() {
dsTypeThreadLocal.remove();
}
}
3.4. IService interface
This interface acts as a flag. When a class needs to enable read-write separation, it needs to implement this interface. The class that implements this interface will be intercepted by the read-write separation interceptor.
package com.javacode2018.readwritesplit.base;
//需要实现读写分离的service需要实现该接口
public interface IService {
}
3.5、ReadWriteDataSource
Read and write separate data sources, inherit ReadWriteDataSource, pay attention to its internal determineCurrentLookupKey method, and obtain the flag of whether to go to the main library or the slave library from the above ThreadLocal.
package com.javacode2018.readwritesplit.base;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
import org.springframework.lang.Nullable;
public class ReadWriteDataSource extends AbstractRoutingDataSource {
@Nullable
@Override
protected Object determineCurrentLookupKey() {
return DsTypeHolder.getDsType();
}
}
3.6、ReadWriteInterceptor
The read-write separation interceptor needs to be executed before the transaction interceptor. Through the @1 code, we set the order of this interceptor to Integer.MAX_VALUE - 2, and later we set the order of the transaction interceptor to Integer.MAX_VALUE - 1 , the execution order of the transaction interceptor is from small to small, so ReadWriteInterceptor will be executed before the transaction interceptor org.springframework.transaction.interceptor.TransactionInterceptor.
Since there are mutual calls in the business methods, for example, service2.m2 is called in service1.m1, and service2.m3 is called in service2.m2, we only need to obtain the specific data source to be used before the m1 method is executed. , so the following code will record whether to go to the main library or the slave library when entering the interceptor for the first time.
The following method will get the last parameter of the current target method. The last parameter can be of type DsType. Developers can use this parameter to control whether to use the main library or the slave library.
package com.javacode2018.readwritesplit.base;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import java.util.Objects;
@Aspect
@Order(Integer.MAX_VALUE - 2) //@1
@Component
public class ReadWriteInterceptor {
@Pointcut("target(IService)")
public void pointcut() {
}
//获取当前目标方法的最后一个参数
private Object getLastArgs(final ProceedingJoinPoint pjp) {
Object[] args = pjp.getArgs();
if (Objects.nonNull(args) && args.length > 0) {
return args[args.length - 1];
} else {
return null;
}
}
@Around("pointcut()")
public Object around(final ProceedingJoinPoint pjp) throws Throwable {
//判断是否是第一次进来,用于处理事务嵌套
boolean isFirst = false;
try {
if (DsTypeHolder.getDsType() == null) {
isFirst = true;
}
if (isFirst) {
Object lastArgs = getLastArgs(pjp);
if (DsType.SLAVE.equals(lastArgs)) {
DsTypeHolder.slave();
} else {
DsTypeHolder.master();
}
}
return pjp.proceed();
} finally {
//退出的时候,清理
if (isFirst) {
DsTypeHolder.clearDsType();
}
}
}
}
3.7、ReadWriteConfiguration
Spring configuration class, function
1. @3: Used to register some classes in the com.javacode2018.readwritesplit.base package into the spring container, such as the above interceptor ReadWriteInterceptor
2. @1: Enable the function of spring aop
3. @2: Enable the function of spring to automatically manage transactions. The order of @EnableTransactionManagement is used to specify the order of the transaction interceptor org.springframework.transaction.interceptor.TransactionInterceptor. Here we set the order to Integer.MAX_VALUE - 1, and the above ReadWriteInterceptor The order is Integer.MAX_VALUE - 2, so the ReadWriteInterceptor will be executed before the transaction interceptor.
package com.javacode2018.readwritesplit.base;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.transaction.annotation.EnableTransactionManagement;
@Configuration
@EnableAspectJAutoProxy //@1
@EnableTransactionManagement(proxyTargetClass = true, order = Integer.MAX_VALUE - 1) //@2
@ComponentScan(basePackageClasses = IService.class) //@3
public class ReadWriteConfiguration {
}
3.8、@EnableReadWrite
This annotation uses two to enable the function of read-write separation. @1 imports ReadWriteConfiguration into the spring container through @Import, which will automatically enable the function of read-write separation. If you need to use read-write separation in your business, you only need to add the @EnableReadWrite annotation to the spring configuration class.
package com.javacode2018.readwritesplit.base;
import org.springframework.context.annotation.Import;
import java.lang.annotation.*;
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Import(ReadWriteConfiguration.class) //@1
public @interface EnableReadWrite {
}
4. Case
The key code for reading and writing separation is finished, let's use a case to verify the effect.
4.1, execute sql script
Two databases are prepared below: javacode2018_master (main library), javacode2018_slave (slave library)
A t_user table is created in both databases, and a piece of data is inserted respectively. Later, this data is used to verify whether it is the master database or the slave database.
DROP DATABASE IF EXISTS javacode2018_master;
CREATE DATABASE IF NOT EXISTS javacode2018_master;
USE javacode2018_master;
DROP TABLE IF EXISTS t_user;
CREATE TABLE t_user (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(256) NOT NULL DEFAULT ''
COMMENT '姓名'
);
INSERT INTO t_user (name) VALUE ('master库');
DROP DATABASE IF EXISTS javacode2018_slave;
CREATE DATABASE IF NOT EXISTS javacode2018_slave;
USE javacode2018_slave;
DROP TABLE IF EXISTS t_user;
CREATE TABLE t_user (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(256) NOT NULL DEFAULT ''
COMMENT '姓名'
);
INSERT INTO t_user (name) VALUE ('slave库');
4.2, spring configuration class
@1: Enable read-write separation
masterDs() method: define the main database data source
slaveDs() method: define slave data source
dataSource(): Define the read-write separation routing data source
There are two more methods to define the JdbcTemplate and the transaction manager. In the method, @Qualifier("dataSource") is used to limit the name of the injected bean to dataSource: that is, the read-write separation routing data source returned by the above dataSource() is injected .
package com.javacode2018.readwritesplit.demo1;
import com.javacode2018.readwritesplit.base.DsType;
import com.javacode2018.readwritesplit.base.EnableReadWrite;
import com.javacode2018.readwritesplit.base.ReadWriteDataSource;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;
@EnableReadWrite //@1
@Configuration
@ComponentScan
public class MainConfig {
//主库数据源
@Bean
public DataSource masterDs() {
org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_master?characterEncoding=UTF-8");
dataSource.setUsername("root");
dataSource.setPassword("root123");
dataSource.setInitialSize(5);
return dataSource;
}
//从库数据源
@Bean
public DataSource slaveDs() {
org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_slave?characterEncoding=UTF-8");
dataSource.setUsername("root");
dataSource.setPassword("root123");
dataSource.setInitialSize(5);
return dataSource;
}
//读写分离路由数据源
@Bean
public ReadWriteDataSource dataSource() {
ReadWriteDataSource dataSource = new ReadWriteDataSource();
//设置主库为默认的库,当路由的时候没有在datasource那个map中找到对应的数据源的时候,会使用这个默认的数据源
dataSource.setDefaultTargetDataSource(this.masterDs());
//设置多个目标库
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put(DsType.MASTER, this.masterDs());
targetDataSources.put(DsType.SLAVE, this.slaveDs());
dataSource.setTargetDataSources(targetDataSources);
return dataSource;
}
//JdbcTemplate,dataSource为上面定义的注入读写分离的数据源
@Bean
public JdbcTemplate jdbcTemplate(@Qualifier("dataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
//定义事务管理器,dataSource为上面定义的注入读写分离的数据源
@Bean
public PlatformTransactionManager transactionManager(@Qualifier("dataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
}
4.3、UserService
This class is equivalent to the service we usually write. I directly use JdbcTemplate to operate the database for the method, and the real project operation db will be placed in dao.
getUserNameById method: query name by id.
insert method: Insert data. All internal operations will go to the main database. In order to verify whether the query will also go to the main database. After inserting the data, we will call the this.userService.getUserNameById(id, DsType.SLAVE) method to execute the query Operation, the second parameter uses SLAVE intentionally. If the query has results, it means that the master library is used, otherwise the slave library is used. Why do you need to call getUserNameById through this.userService here?
this.userService is ultimately a proxy object, accessing its internal methods through the proxy object will be intercepted by the interceptor that separates reading and writing.
package com.javacode2018.readwritesplit.demo1;
import com.javacode2018.readwritesplit.base.DsType;
import com.javacode2018.readwritesplit.base.IService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import java.util.List;
@Component
public class UserService implements IService {
@Autowired
private JdbcTemplate jdbcTemplate;
@Autowired
private UserService userService;
@Transactional(propagation = Propagation.SUPPORTS, readOnly = true)
public String getUserNameById(long id, DsType dsType) {
String sql = "select name from t_user where id=?";
List<String> list = this.jdbcTemplate.queryForList(sql, String.class, id);
return (list != null && list.size() > 0) ? list.get(0) : null;
}
//这个insert方法会走主库,内部的所有操作都会走主库
@Transactional
public void insert(long id, String name) {
System.out.println(String.format("插入数据{id:%s, name:%s}", id, name));
this.jdbcTemplate.update("insert into t_user (id,name) values (?,?)", id, name);
String userName = this.userService.getUserNameById(id, DsType.SLAVE);
System.out.println("查询结果:" + userName);
}
}
4.4. Test cases
package com.javacode2018.readwritesplit.demo1;
import com.javacode2018.readwritesplit.base.DsType;
import org.junit.Before;
import org.junit.Test;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
public class Demo1Test {
UserService userService;
@Before
public void before() {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(MainConfig.class);
context.refresh();
this.userService = context.getBean(UserService.class);
}
@Test
public void test1() {
System.out.println(this.userService.getUserNameById(1, DsType.MASTER));
System.out.println(this.userService.getUserNameById(1, DsType.SLAVE));
}
@Test
public void test2() {
long id = System.currentTimeMillis();
System.out.println(id);
this.userService.insert(id, "张三");
}
}
The test1 method executes two queries, querying the main library and the slave library respectively, and outputs:
master库
slave库
Isn't it cool, the developers themselves control whether to use the main library or the slave library.
The execution results of test2 are as follows. It can be seen that the data just inserted has been queried, indicating that all operations in insert go through the main database.
1604905117467
插入数据{id:1604905117467, name:张三}
查询结果:张三
5. Case source code
git地址:
https://gitee.com/javacode2018/spring-series
本文案例对应源码:
spring-series\lesson-004-readwritesplit
Everyone star it, all series of codes will be in this.
Source: https://mp.weixin.qq.com/s?__biz=MzA5MTkxMDQ4MQ==&mid=2648938118&idx=2&sn=baef96540a8936e49db0bfe62f909f24&scene=21#wechat_redirect
ster library
slave library
是不是很爽,由开发者自己控制具体走主库还是从库。
test2 执行结果如下,可以看出查询到了刚刚插入的数据,说明 insert 中所有操作都走的是主库。
1604905117467
Insert data {id: 1604905117467, name: Zhang San}
Query result: Zhang San
## 5、案例源码
Git address:
https://gitee.com/javacode2018/spring-series
The source code corresponding to the case in this article:
spring-series\lesson-004-readwritesplit
大家 star 一下,所有系列代码都会在这个里面。
来源:https://mp.weixin.qq.com/s?__biz=MzA5MTkxMDQ4MQ==&mid=2648938118&idx=2&sn=baef96540a8936e49db0bfe62f909f24&scene=21#wechat_redirect