Using a database table, data processing multiple data sources

Spring Boot project through the interface dynamically add / remove a data source, and then add a data source, dynamically switch data sources, and then use the data source after switching mybaties inquiry.

1. First, the data source necessary to create a table in the database, the information stored in the connection information table in the database.

#数据源信息表
DROP TABLE IF EXISTS `datasource_config`;
CREATE TABLE IF NOT EXISTS `datasource_config`
(
    `id`       bigint(13)   NOT NULL AUTO_INCREMENT COMMENT '主键',
    `host`     varchar(255) NOT NULL COMMENT '数据库地址',
    `port`     int(6)       NOT NULL COMMENT '数据库端口',
    `username` varchar(100) NOT NULL COMMENT '数据库用户名',
    `password` varchar(100) NOT NULL COMMENT '数据库密码',
    `database` varchar(100) DEFAULT 0 COMMENT '数据库名称',
    PRIMARY KEY (`id`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8 COMMENT ='数据源配置表';

2. Main Code

2.1 pom file

 pom section information file needs to add dependencies and data dependencies connection

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-aop</artifactId>
    </dependency>

    <dependency>
      <groupId>tk.mybatis</groupId>
      <artifactId>mapper-spring-boot-starter</artifactId>
      <version>2.1.5</version>
    </dependency>

    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <scope>runtime</scope>
    </dependency>

2.2 Basic Configuration class

DatasourceConfig, mainly to build his own definition of a data source by DataSourceBuilder, which is placed in a container Spring

@Configuration
public class DatasourceConfiguration {

    @Bean
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource dataSource() {
        DataSourceBuilder<?> dataSourceBuilder = DataSourceBuilder.create();
        dataSourceBuilder.type(DynamicDataSource.class);
        return dataSourceBuilder.build();
    }
}

MybatisConfiguration mainly to build out the previous step to Mybatis data source configuration in the `SqlSessionFactory`

@Configuration
@MapperScan(basePackages = "data.cloud.datas.mapper", sqlSessionFactoryRef = "sqlSessionFactory")
public class MybatisConfiguration {
    /**
     * 创建会话工厂。
     *
     * @param dataSource 数据源
     * @return 会话工厂
     */
    @Bean(name = "sqlSessionFactory")
    @SneakyThrows
    public SqlSessionFactory getSqlSessionFactory(@Qualifier("dataSource") DataSource dataSource) throws Exception {
        SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
        bean.setDataSource(dataSource);
        return bean.getObject();
    }
}

The main logic of dynamic data sources 2.3

DatasourceConfigContextHolder primary data source id for binding threads that are currently used to ensure that the same thread through ThreadLocal not be modified

public class DatasourceConfigContextHolder {
    private static final ThreadLocal<Long> DATASOURCE_HOLDER = ThreadLocal.withInitial(() -> DatasourceHolder.DEFAULT_ID);

    private static Long temp;
    /**
     * 设置默认数据源
     */
    public static void setDefaultDatasource() {
        DATASOURCE_HOLDER.remove();
        setCurrentDatasourceConfig(DatasourceHolder.DEFAULT_ID);
    }

    /**
     * 获取当前数据源配置id
     */
    public static Long getCurrentDatasourceConfig() {
        //return DATASOURCE_HOLDER.get();
        return temp;
    }

    /**
     * 设置当前数据源配置id
     */
    public static void setCurrentDatasourceConfig(Long id) {
        DATASOURCE_HOLDER.set(id);
        temp = id;
    }
}

DynamicDataSource, class inherits `com.zaxxer.hikari.HikariDataSource`, mainly for dynamically switching data source connection.

@Slf4j
public class DynamicDataSource extends HikariDataSource {
    @Override
    public Connection getConnection() throws SQLException {
        // 获取当前数据源 id
        Long id = DatasourceConfigContextHolder.getCurrentDatasourceConfig();
        // 根据当前id获取数据源
        HikariDataSource datasource = DatasourceHolder.INSTANCE.getDatasource(id);
        if (null == datasource) {
            datasource = initDatasource(id);
        }
        return datasource.getConnection();
    }
    /**
     * 初始化数据源
     * @param id 数据源id
     * @return 数据源
     */
    private HikariDataSource initDatasource(Long id) {
        HikariDataSource dataSource = new HikariDataSource();

        // 判断是否是默认数据源
        if (DatasourceHolder.DEFAULT_ID.equals(id)) {
            // 默认数据源根据 application.yml 配置的生成
            DataSourceProperties properties = SpringUtil.getBean(DataSourceProperties.class);
            dataSource.setJdbcUrl(properties.getUrl());
            dataSource.setUsername(properties.getUsername());
            dataSource.setPassword(properties.getPassword());
            dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
        } else {
            // 不是默认数据源，通过缓存获取对应id的数据源的配置
            SourceConfig sourceConfig = DatasourceConfigCache.INSTANCE.getConfig(id);
            if (sourceConfig == null) {
                throw new RuntimeException("无此数据源");
            }
            dataSource.setJdbcUrl(sourceConfig.buildJdbcUrl());
            dataSource.setUsername(sourceConfig.getUsername());
            dataSource.setPassword(sourceConfig.getPassword());
            dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
        }
        // 将创建的数据源添加到数据源管理器中，绑定当前线程
        DatasourceHolder.INSTANCE.addDatasource(id, dataSource);
        return dataSource;
    }
}

DatasourceScheduler mainly used for the scheduling task

public enum DatasourceScheduler {
    /**
     * 当前实例
     */
    INSTANCE;
    private AtomicInteger cacheTaskNumber = new AtomicInteger(1);
    private ScheduledExecutorService scheduler;
    DatasourceScheduler() {
        create();
    }
    private void create() {
        this.shutdown();
        this.scheduler = new ScheduledThreadPoolExecutor(10, r -> new Thread(r, String.format("Datasource-Release-Task-%s", cacheTaskNumber.getAndIncrement())));
    }
    private void shutdown() {
        if (null != this.scheduler) {
            this.scheduler.shutdown();
        }
    }
    public void schedule(Runnable task,long delay){
        this.scheduler.scheduleAtFixedRate(task, delay, delay, TimeUnit.MILLISECONDS);
    }
}

DatasourceManager, primarily used to manage data sources, data source recorded last used time and determines whether or not used for a long time, over a certain time is not used, the connection will be released.

public class DatasourceManager {
    //默认释放时间
 
    private static final Long DEFAULT_RELEASE = 10L;
    //数据源
    @Getter
    private HikariDataSource dataSource;
    public HikariDataSource getDataSource() {
        return dataSource;
    }
    public void setDataSource(HikariDataSource dataSource) {
        this.dataSource = dataSource;
    }
    /**
     * 上一次使用时间
     */
    private LocalDateTime lastUseTime;
    public DatasourceManager(HikariDataSource dataSource) {
        this.dataSource = dataSource;
        this.lastUseTime = LocalDateTime.now();
    }
    /**
     * 是否已过期，如果过期则关闭数据源
     * @return 是否过期，{@code true} 过期，{@code false} 未过期
     */
    public boolean isExpired() {
        if (LocalDateTime.now().isBefore(this.lastUseTime.plusMinutes(DEFAULT_RELEASE))) {
            return false;
        }
        this.dataSource.close();
        return true;
    }
    /**
     * 刷新上次使用时间
     */
    public void refreshTime() {
        this.lastUseTime = LocalDateTime.now();
    }
}

DatasourceHolder, class is mainly used to manage the data source, through `DatasourceScheduler` regularly check whether the data source is not used for a long time, it releases the connection timeout.

public enum DatasourceHolder {
    //当前实例
    INSTANCE;
    //启动执行，定时5分钟清理一次
    DatasourceHolder() {
        DatasourceScheduler.INSTANCE.schedule(this::clearExpiredDatasource, 5 * 60 * 1000);
    }
    /**
     * 默认数据源的id
     */
    public static final Long DEFAULT_ID = -1L;
    /**
     * 管理动态数据源列表。
     */
    private static final Map<Long, DatasourceManager> DATASOURCE_CACHE = new ConcurrentHashMap<>();
    /**
     * 添加动态数据源
     * @param id         数据源id
     * @param dataSource 数据源
     */
    public synchronized void addDatasource(Long id, HikariDataSource dataSource) {
        DatasourceManager datasourceManager = new DatasourceManager(dataSource);
        DATASOURCE_CACHE.put(id, datasourceManager);
    }
    /**
     * 查询动态数据源
     * @param id 数据源id
     * @return 数据源
     */
    public synchronized HikariDataSource getDatasource(Long id) {
        if (DATASOURCE_CACHE.containsKey(id)) {
            DatasourceManager datasourceManager = DATASOURCE_CACHE.get(id);
            datasourceManager.refreshTime();
            return datasourceManager.getDataSource();
        }
        return null;
    }
    /**
     * 清除超时的数据源
     */
    public synchronized void clearExpiredDatasource() {
        DATASOURCE_CACHE.forEach((k, v) -> {
            // 排除默认数据源
            if (!DEFAULT_ID.equals(k)) {
                if (v.isExpired()) {
                    DATASOURCE_CACHE.remove(k);
                }
            }
        });
    }
    /**
     * 清除动态数据源
     * @param id 数据源id
     */
    public synchronized void removeDatasource(Long id) {
        if (DATASOURCE_CACHE.containsKey(id)) {
            // 关闭数据源
            DATASOURCE_CACHE.get(id).getDataSource().close();
            // 移除缓存
            DATASOURCE_CACHE.remove(id);
        }
    }
}

DatasourceConfigCache，该类主要用于缓存数据源的配置，用户生成数据源时，获取数据源连接参数。

public enum DatasourceConfigCache {
    /**
     * 当前实例
     */
    INSTANCE;
    /**
     * 管理动态数据源列表。
     */
    private static final Map<Long, SourceConfig> CONFIG_CACHE = new ConcurrentHashMap<>();
    /**
     * 添加数据源配置
     */
    public synchronized void addConfig(Long id, SourceConfig config) {
        CONFIG_CACHE.put(id, config);
    }
    /**
     * 查询数据源配置
     */
    public synchronized SourceConfig getConfig(Long id) {
        if (CONFIG_CACHE.containsKey(id)) {
            return CONFIG_CACHE.get(id);
        }
        return null;
    }
    // 清除数据源配置
    public synchronized void removeConfig(Long id) {
        CONFIG_CACHE.remove(id);
        // 同步清除 DatasourceHolder 对应的数据源
        DatasourceHolder.INSTANCE.removeDatasource(id);
    }
}

2.4 default data source configuration and switching data source

DefaultDatasource，默认数据源

//默认数据源
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface DefaultDatasource {
}

DatasourceSelectorAspect，主要用于切换数据源

@Aspect
@Component
public class DatasourceSelectorAspect {
    @Pointcut("execution(public * data.cloud.datas.controller.*.*(..))")
    public void datasourcePointcut(){
    }
    /**
     * 前置操作，拦截具体请求，获取header里的数据源id，设置线程变量里，用于后续切换数据源
     */
    @Before("datasourcePointcut()")
    public void doBefore(JoinPoint joinPoint) {
        Signature signature = joinPoint.getSignature();
        MethodSignature methodSignature = (MethodSignature) signature;
        Method method = methodSignature.getMethod();
        // 排除不可切换数据源的方法
        DefaultDatasource annotation = method.getAnnotation(DefaultDatasource.class);
        if (null != annotation) {
            DatasourceConfigContextHolder.setDefaultDatasource();
        } else {
            RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
            ServletRequestAttributes attributes = (ServletRequestAttributes) requestAttributes;
            HttpServletRequest request = attributes.getRequest();
            String configIdInHeader = request.getHeader("Datasource-Config-Id");
            if (StringUtils.hasText(configIdInHeader)) {
                long configId = Long.parseLong(configIdInHeader);
                DatasourceConfigContextHolder.setCurrentDatasourceConfig(configId);
            } else {
                DatasourceConfigContextHolder.setDefaultDatasource();
            }
        }
    }
    /**
     * 后置操作，设置回默认的数据源id
     */
    @AfterReturning("datasourcePointcut()")
    public void doAfter() {
        DatasourceConfigContextHolder.setDefaultDatasource();
    }
}

2.5 Tools

SpringUtils

@Slf4j
@Service
@Lazy(false)
public class SpringUtil implements ApplicationContextAware, DisposableBean {
    private static ApplicationContext applicationContext = null;
    private static Logger log = LoggerFactory.getLogger(SpringUtil.class);
    //取得存储在静态变量中的ApplicationContext.
    public static ApplicationContext getApplicationContext() {
        return applicationContext;
    }
    //实现ApplicationContextAware接口, 注入Context到静态变量中.
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) {
        SpringUtil.applicationContext = applicationContext;
    }
    //从静态变量applicationContext中取得Bean, 自动转型为所赋值对象的类型.
    @SuppressWarnings("unchecked")
    public static <T> T getBean(String name) {
        return (T) applicationContext.getBean(name);
    }
    //从静态变量applicationContext中取得Bean, 自动转型为所赋值对象的类型.
    public static <T> T getBean(Class<T> requiredType) {
        return applicationContext.getBean(requiredType);
    }
    //清除SpringContextHolder中的ApplicationContext为Null.
    public static void clearHolder() {
        if (log.isDebugEnabled()) {
            log.debug("清除SpringContextHolder中的ApplicationContext:" + applicationContext);
        }
        applicationContext = null;
    }
    /**
     * 发布事件
     * @param event 事件
     */
    public static void publishEvent(ApplicationEvent event) {
        if (applicationContext == null) {
            return;
        }
        applicationContext.publishEvent(event);
    }
    /**
     * 实现DisposableBean接口, 在Context关闭时清理静态变量.
     */
    @Override
    public void destroy() {
        SpringUtil.clearHolder();
    }
}

2.6 startup class

After startup, use the default data source query data source configuration list, its cache to `DatasourceConfigCache` in, for later use.

@SpringBootApplication
@EnableSwagger2
@EnableTransactionManagement
public class DatasApplication implements CommandLineRunner {
    @Resource
    private ConfigMapper configMapper;
    public static void main(String[] args) {
        SpringApplication.run(DatasApplication.class, args);
    }
    @Override
    public void run(String... args) {
        // 设置默认的数据源
        DatasourceConfigContextHolder.setDefaultDatasource();
        // 查询所有数据库配置列表
        List<SourceConfig> sourceConfigs = configMapper.selectAll();
        System.out.println("加载其余数据源配置列表: " + sourceConfigs);
        // 将数据库配置加入缓存
        sourceConfigs.forEach(config -> DatasourceConfigCache.INSTANCE.addConfig(config.getId(), config));
    }
}

3. Test

Add Value Datasource-Config-Id header in the request header, the value is the id value of data in the source table