springboot学习(五):MyBatis动态数据源配置

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/sinat_36553913/article/details/82527099

说明

通过上篇博文《MyBatis多数据源配置》,学习了MyBatis多数据源的配置。在实际环境中,数据库一般配置为主从结构的形式,甚至是一主多从的形式,这时,我们希望读操作都在从数据库中进行,增删改操作都在主数据库中进行,希望mybatis能够动态地选择数据库。通过本篇博文,学习记录下通过注解+AOP和只使用AOP两种方式实现动态数据源配置。文中使用主机的mysql作为主数据库,虚机中的mysql作为从数据库。

正文

1.通过spirngboot构建项目

主要依赖 mybatis+mysql+aspect

<dependency>
    <groupId>org.mybatis.spring.boot</groupId>
    <artifactId>mybatis-spring-boot-starter</artifactId>
    <version>1.3.2</version>
</dependency>
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
    <version>1.8.7</version>
</dependency>

2.在application.properties配置数据库

#主库
datasource.master.driverClassName=com.mysql.jdbc.Driver
datasource.master.url=jdbc:mysql://ip:3306/dbname?characterEncoding=UTF-8
datasource.master.username=**
datasource.master.password=**

#从库
datasource.slave.driverClassName=com.mysql.jdbc.Driver
datasource.slave.url=jdbc:mysql://ip:3306/dbname?characterEncoding=UTF-8
datasource.slave.username=**
datasource.slave.password=**

3.在启动类关闭自动配置数据源

这个很重要,如果没有关闭,在启动程序时会发生循环依赖的错误

@EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class})
@SpringBootApplication
public class DynamicDatasourceAnnoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DynamicDatasourceAnnoApplication.class, args);
    }
}

4.配置数据源

要实现数据源的动态选择,AbstractRoutingDataSource这个抽象类很重要,通过阅读源码,发现数据源在该类中以key-value的形式存储在map中,其中在determineTargetDataSource()方法中,通过determineCurrentLookupKey()方法得到key,在map中寻找需要的数据源,而determineCurrentLookupKey()是抽象方法,我们可以通过继承这个抽象类实现该方法,配置我们期望的数据源。

public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBean {
protected DataSource determineTargetDataSource() {
        Assert.notNull(this.resolvedDataSources, "DataSource router not initialized");
        Object lookupKey = this.determineCurrentLookupKey();
        DataSource dataSource = (DataSource)this.resolvedDataSources.get(lookupKey);
        if (dataSource == null && (this.lenientFallback || lookupKey == null)) {
            dataSource = this.resolvedDefaultDataSource;
        }

        if (dataSource == null) {
            throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]");
        } else {
            return dataSource;
        }
}
protected abstract Object determineCurrentLookupKey();

在了解了基本原理后,开始配置数据源,继承AbstractRoutingDataSource 类

DynamicDataSource

public class DynamicDataSource extends AbstractRoutingDataSource {
    @Override
    protected Object determineCurrentLookupKey() {
        return DynamicDataSourceHolder.getType();
    }
}

实现了该方法后,如何返回一个key,这个key从何而来?
每个请求都是一个线程,我们可以再创建一个类,其中使用ThreadLocal线程局部变量保存每个线程所请求方法对应的key,通过该类的静态方法返回该请求的key

DynamicDataSourceHolder

public class DynamicDataSourceHolder {
    private static final ThreadLocal<DataSourceType> dataSourceHolder = new ThreadLocal<>();

    private static final Set<DataSourceType> dataSourceTypes = new HashSet<>();
    static {
        dataSourceTypes.add(DataSourceType.MASTER);
        dataSourceTypes.add(DataSourceType.SLAVE);
    }

    public static void setType(DataSourceType dataSourceType){
        if(dataSourceType == null){
            throw new NullPointerException();
        }
        dataSourceHolder.set(dataSourceType);
    }

    public static DataSourceType getType(){
        return dataSourceHolder.get();
    }

    public static void clearType(){
        dataSourceHolder.remove();
    }

    public static boolean containsType(DataSourceType dataSourceType){
        return dataSourceTypes.contains(dataSourceType);
    }
}

其中使用了枚举表示不同类型的数据源

DataSourceType

public enum DataSourceType {
    MASTER,
    SLAVE,
}

创建了关键类DynamicDataSource后,开始配置数据源

DynamicDataSourceConfig

@Configuration
public class DynamicDataSourceConfig {

    @Bean(name = "masterDataSource")
    @ConfigurationProperties(prefix = "datasource.master")
    public DataSource masterDataSource(){
        return DataSourceBuilder.create().build();
    }

    @Bean(name = "slaveDataSource")
    @ConfigurationProperties(prefix = "datasource.slave")
    public DataSource slaveDataSource(){
        return DataSourceBuilder.create().build();
    }

    @Bean(name = "dynamicDataSource")
    @Primary
    public DataSource dynamicDataSource(@Qualifier(value = "masterDataSource") DataSource masterDataSource,
                                        @Qualifier(value = "slaveDataSource") DataSource slaveDataSource){
        DynamicDataSource dynamicDataSource = new DynamicDataSource();
        dynamicDataSource.setDefaultTargetDataSource(masterDataSource);
        Map<Object,Object> dataSourceMap = new HashMap<>();
        dataSourceMap.put(DataSourceType.MASTER,masterDataSource);
        dataSourceMap.put(DataSourceType.SLAVE,slaveDataSource);
        dynamicDataSource.setTargetDataSources(dataSourceMap);
        return dynamicDataSource;
    }

}

配置好数据源后,开始配置mybatis

MyBatisConfig

@Configuration
@MapperScan(basePackages = "com.example.dynamic_datasource_anno.db.dao")
public class MyBatisConfig {

    @Autowired
    private DataSource dataSource;

    @Bean(name = "sqlSessionFactory")
    public SqlSessionFactory sqlSessionFactory() throws Exception {
        SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean();
        sessionFactoryBean.setDataSource(dataSource);
        sessionFactoryBean.setTypeAliasesPackage("com.example.dynamic_datasource_anno.db.pojo");
        Resource[] resources = new PathMatchingResourcePatternResolver()
                .getResources("classpath:com/example/dynamic_datasource_anno/db/mapper/*Mapper.xml");
        sessionFactoryBean.setMapperLocations(resources);

        return sessionFactoryBean.getObject();
    }

}

数据源和mybatis配置好后,开始配置切面,动态地选择都在此类中完成。

第一种 Anno+Aspect

创建注解DBType,默认为主数据库

扫描二维码关注公众号,回复: 3364460 查看本文章
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface DBType {
    DataSourceType value() default DataSourceType.MASTER;
}

切面DynamicDataSourceAspect

@Aspect
@Component
public class DynamicDataSourceAspect {

    private static final Logger logger = LoggerFactory.getLogger(DynamicDataSourceAspect.class);

    @Before("@annotation(dbType)")
    public void changeDataSourceType(JoinPoint joinPoint, DBType dbType){
        DataSourceType curType = dbType.value();
        if(!DynamicDataSourceHolder.containsType(curType)){
            logger.info("指定数据源[{}]不存在,使用默认数据源-> {}",dbType.value(),joinPoint.getSignature());
        }else{
            logger.info("use datasource {} -> {}",dbType.value(),joinPoint.getSignature());
            DynamicDataSourceHolder.setType(dbType.value());
        }

    }

    @After("@annotation(dbType)")
    public void restoreDataSource(JoinPoint joinPoint, DBType dbType){
        logger.info("use datasource {} -> {}",dbType.value(),joinPoint.getSignature());
        DynamicDataSourceHolder.clearType();
    }

}

配置好切面后,可以在Mapper接口的方法上使用@DBType指定要使用的数据源,使用自动代码生成插件将表中数据生成对应的实体类,mapper文件,mapper接口,关于插件的使用,请看之前的博文《使用c3p0连接池集成mybatis及mybatis自动代码生成插件》
此处,我在service层使用注解

 @DBType(DataSourceType.SLAVE)
 public void select(){
      User user = userMapper.selectByPrimaryKey(1);
      System.out.println(user.getId() + "--" + user.getName() + "==" + user.getGender());
 }

 @DBType(DataSourceType.MASTER)
 public void insert(User user){
      userMapper.insertSelective(user);
 }

第二种 只使用Aspect,判断sql类型

在此方式中,没有再使用枚举、注解,数据源和mybatis的配置与之前相似,不再赘述,详细配置可以查看此方式的源码。主要不同在于切面的配置,此配置中通过切点判断请求的SqlCommandType来选择数据源
DynamicDataSourceAspect

@Aspect
@Component
public class DynamicDataSourceAspect {

    private static final Logger logger = LoggerFactory.getLogger(DynamicDataSourceAspect.class);
    @Autowired
    @Qualifier("sqlSessionFactory")
    private SqlSessionFactory sqlSessionFactory;

    // first * any return value
    // second * any class name of the dao package
    // third * any method name of the class
    // (..) any number of parameters
    // @Pointcut("within(com.example.dynamic_datasource_method.db.dao..*)")
    @Pointcut("execution(* com.example.dynamic_datasource_method.db.dao.*.*(..))")
    public void declareJoinPoint(){
    }

    @Before("declareJoinPoint()")
    public void matchDataSoruce(JoinPoint point){

        //得到被代理对象
        Object target = point.getTarget();
        //目标方法名
        String methodName = point.getSignature().getName();

        Class<?>[] interfaces = target.getClass().getInterfaces();
        Class<?>[] parametersTypes = ((MethodSignature)point.getSignature()).getMethod().getParameterTypes();

        try {
            Method method = interfaces[0].getMethod(methodName,parametersTypes);
            String key = interfaces[0].getName() + "." + method.getName();
            //sql 的类型
            SqlCommandType type = sqlSessionFactory.getConfiguration().getMappedStatement(key).getSqlCommandType();
            //查询从库 增删改主库
            if(type == SqlCommandType.SELECT){
                logger.info("use slaveDataSource");
                DynamicDataSourceHolder.setType("slave");
            }
            if(type == SqlCommandType.DELETE || type == SqlCommandType.UPDATE || type == SqlCommandType.INSERT){
                logger.info("use masterDataSource");
                DynamicDataSourceHolder.setType("master");
            }
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        }
    }

    @After("declareJoinPoint()")
    public void restoreDataSource(JoinPoint joinPoint){
        logger.info(DynamicDataSourceHolder.getType() + " change to master" + joinPoint.getSignature());
        DynamicDataSourceHolder.clearType();
    }
}

通过此方式,可以直接调用mapper接口的方法而不需要注解直接动态地选择数据源

问题

当在启动类没有关闭数据源的自动配置时,启动项目会报以下错误:

Description:

The dependencies of some of the beans in the application context form a cycle:

   dynamicDataSourceAspect (field private org.apache.ibatis.session.SqlSessionFactory com.example.dynamic_datasource_method.config.mybatis.DynamicDataSourceAspect.sqlSessionFactory)
      ↓
   myBatisConfig (field private javax.sql.DataSource com.example.dynamic_datasource_method.config.mybatis.MyBatisConfig.dataSource)
┌─────┐
|  dynamicDataSource defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]
↑     ↓
|  masterDataSource defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]
↑     ↓
|  dataSourceInitializer
└─────┘


2018-09-08 10:04:09.537 ERROR 20372 --- [           main] o.s.test.context.TestContextManager      : Caught exception while allowing TestExecutionListener [org.springframework.test.context.web.ServletTestExecutionListener@68ceda24] to prepare test instance [com.example.dynamic_datasource_method.DynamicDatasourceMethodApplicationTests@65ae095c]

java.lang.IllegalStateException: Failed to load ApplicationContext
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:189) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:131) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:228) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:287) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:289) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:247) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137) [junit-4.12.jar:4.12]
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) [junit-rt.jar:na]
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) [junit-rt.jar:na]
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) [junit-rt.jar:na]
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) [junit-rt.jar:na]
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dynamicDataSourceAspect': Unsatisfied dependency expressed through field 'sqlSessionFactory'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'myBatisConfig': Unsatisfied dependency expressed through field 'dataSource'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dynamicDataSource' defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]: Unsatisfied dependency expressed through method 'dynamicDataSource' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'masterDataSource' defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceInitializer': Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'dynamicDataSource': Requested bean is currently in creation: Is there an unresolvable circular reference?
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1272) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
    at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:121) ~[spring-boot-test-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
    ... 24 common frames omitted

思考

关于最后一种方式判断sql类型达到数据源的动态地选择,此时mysql为一主一从的结构,若为一主多从的结构时如何实现动态选择?以下是大体思路:

本项目实现了一主一从动态数据源的选择,使用切面判断sql方法的方式。

这里的多数据的动态选择,主要是一主一从的形式,若有一主多从的方式,怎么实现数据源的动态选择?

增删改可以使用一个主数据库,查则可以有多个从数据库可以选择,对于多个从数据库,如何选择?

当有多个从数据库时,初步设想使用负载均衡,使用轮询的方式将查询分布到多个从数据库中,那如何实现?

动态数据源的实现是继承了抽象类AbstractRoutingDataSource,实现了其中的抽象方法 determineCurrentLookupKey()
通过方法名可知,该方法返回一个key,通过阅读AbstractRoutingDataSource的源码,多个数据源的存储方式
是以key-value的形式存储在map中,当动态选择时,使用determineTargetDataSourc()根据key找到对应的DataSource。

现在要实现多个数据库,设想继承AbstractRoutingDataSource类,重写determineTargetDataSource()方法,将原来Map<Object, Object> targetDataSources的存储方式改为Map<Object,List<Object>> slaveDataSources 用来存储从库数据源,当动态选择时,在determineTargetDataSource()方法中 判断 determineCurrentLookupKey()方法的key,若为master,则直接返回resolvedDefaultDataSource,(前提是在配置数据源时,必须将主数据设置为defaultDataSource).若为slave,则得到slave的List<DataSource>,使用轮询的方式得到其中一个DataSource返回,实现多从的动态选择。

项目源码:
第一种:https://github.com/Edenwds/springboot_study/tree/master/dynamic_datasource_anno
第二种:https://github.com/Edenwds/springboot_study/tree/master/dynamic_datasource_method

参考资料:
https://blog.csdn.net/neosmith/article/details/61202084
https://www.codeblogbt.com/archives/153425
http://fedulov.website/2015/10/14/dynamic-datasource-routing-with-spring/
https://stackoverflow.com/questions/47970429/query-is-always-execute-before-than-aop-in-springboot-and-mybatis-application-fo

猜你喜欢

转载自blog.csdn.net/sinat_36553913/article/details/82527099