今日指数项目集成分库分表

1、股票数据库表拆分规划

1.1 数据分库分表规划

基于上一小节的分析,我们得出一些结论:

  • 对于股票流水表按照月维度和年维护进行库表拆分,也就是说一年会产生一个库用于后期数据归档,而每个库下则按照月份产生12张表,对应一年的数据;
  • 对于板块表和大盘数据表,我们则以年为单位,与股票流水表年份一致即可,也就是按照年分库分表;
  • 对于主营业务表,因为数据量较少,且查询都会用到,作为公共表处理;
  • 对于系统表数据量相对较少,作为默认数据源即可;

整体架构如下:

在这里插入图片描述

数据详见:讲义\v-2\资料\今日指数分库分表SQL脚本

综上,我们以日期时间字段cur_time作为库表的分偏键,分库分表的逻辑存在一定的复杂性,采用标准分片策略比较合适。

1.2 分库分表策略规划

​ 经过分析发现大盘、板块、股票相关数据的分库策略是一致的,而分表策略则存在部分差异,所以我们可先定义公共的分库算法类和公共的分表算法类,对于不一致的,则个别定义即可:

公共分库算法 公共分表算法 说明
stock_block_rt_inf
stock_market_index_info
stock_outer_market_index_info
stock_rt_info 根据月份分表
stock_business 公共表|广播表
系统管理相关表:sys_user等 默认数据源

2、默认数据源配置

2.1 工程依赖准备

在stock_common工程导入sharding-jdbc依赖:

<!--引入shardingjdbc依赖-->
<dependency>
  <groupId>org.apache.shardingsphere</groupId>
  <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
</dependency>

2.2 配置默认数据源

系统管理相关的表数据量较少,无需分库分表,所以可作为sharding-jdbc的默认数据源;

在stock_backend工程配置application-sharding.properties:

# 数据源名称,多数据源以逗号分隔
# 注意事项:数据源名称如果存在多个单词,不要使用小驼峰,建议使用中划线间隔(不要使用下划线间隔符)
spring.shardingsphere.datasource.names=df
# 配置默认数据源
# 数据库连接池类名称
spring.shardingsphere.datasource.df.type=com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
spring.shardingsphere.datasource.df.driver-class-name=com.mysql.jdbc.Driver
# 数据库 url 连接
spring.shardingsphere.datasource.df.url=jdbc:mysql://192.168.200.132:3306/stock_sys_db?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
# 数据库用户名
spring.shardingsphere.datasource.df.username=root
# 数据库密码
spring.shardingsphere.datasource.df.password=root
# 配置默认数据源
spring.shardingsphere.sharding.default-data-source-name=df
# 是否开启 SQL 显示,默认值: false
spring.shardingsphere.props.sql.show=true

主配置文件application.yml激活application-sharding.properties配置:

spring.profiles.active=sharding

同时在application.yml主配置文件中注释掉原有数据源信息!!

2.3 默认数据源效果测试

@SpringBootTest
public class TestSharding {
    
    
    @Autowired
    private SysUserMapper sysUserMapper;
    /**
     * 测试默认数据源
     */
    @Test
    public void testDefaultDs(){
    
    
        SysUser user = sysUserMapper.selectByPrimaryKey("1237365636208922624");
        System.out.println(user);
    }
}

测试时发现报异常错误:

在这里插入图片描述

所以在主配置文件中,配置bean允许被覆盖:

spring.main.allow-bean-definition-overriding=true

最终效果:

在这里插入图片描述

3、广播表配置

对于stock_business业务表作为广播表处理:

在这里插入图片描述

在application-sharding.properties配置其它数据源:

# 第一步:配置shardingjdbc
# 数据源名称,多数据源以逗号分隔(datasource名称不要使用特殊符号)
spring.shardingsphere.datasource.names=ds-2021,ds-2022,df
# 数据库连接池类名称
spring.shardingsphere.datasource.ds-2021.type=com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
spring.shardingsphere.datasource.ds-2021.driver-class-name=com.mysql.jdbc.Driver
# 数据库 url 连接
spring.shardingsphere.datasource.ds-2021.url=jdbc:mysql://192.168.200.130:3306/stock_db_2021?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
# 数据库用户名
spring.shardingsphere.datasource.ds-2021.username=root
# 数据库密码
spring.shardingsphere.datasource.ds-2021.password=root

# 数据库连接池类名称
spring.shardingsphere.datasource.ds-2022.type=com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
spring.shardingsphere.datasource.ds-2022.driver-class-name=com.mysql.jdbc.Driver
# 数据库 url 连接
spring.shardingsphere.datasource.ds-2022.url=jdbc:mysql://192.168.200.131:3306/stock_db_2022?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
# 数据库用户名
spring.shardingsphere.datasource.ds-2022.username=root
# 数据库密码
spring.shardingsphere.datasource.ds-2022.password=root

# 配置默认数据源
# 数据库连接池类名称
spring.shardingsphere.datasource.df.type=com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
spring.shardingsphere.datasource.df.driver-class-name=com.mysql.jdbc.Driver
# 数据库 url 连接
spring.shardingsphere.datasource.df.url=jdbc:mysql://192.168.200.132:3306/stock_sys_db?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
# 数据库用户名
spring.shardingsphere.datasource.df.username=root
# 数据库密码
spring.shardingsphere.datasource.df.password=root

# 配置广播表,如果有多个,以逗号间隔
spring.shardingsphere.sharding.broadcast-tables=stock_business
# 配置默认数据源
spring.shardingsphere.sharding.default-data-source-name=df

# 是否开启 SQL 显示,默认值: false
spring.shardingsphere.props.sql.show=true

测试广播表:

@Autowired
private StockBusinessMapper stockBusinessMapper;
/**
     * @Description 测试广播表
     */
@Test
public void testBroadCastTable(){
    
    
  List<StockBusiness> all = stockBusinessMapper.getAll();
  System.out.println(all);
}

4、大盘板块分库分表实现

大盘板块分库分表思路分析:

  • 对于stock_block_rt_info等相关表一年产出的数据量不大,所以对这类表只做分库处理,而库内无需做分表处理;
  • 大盘板块相关表的分库策略是相同的,所以我们可将分库分表算法抽取出来作为公共算法类,同时库内没有做分表处理,所以无需定义分表策略;
  • 主业务工程和定时任务工程都需要分库或分表的算法类,所以我们在common工程下维护;

4.1 定义公共分库算法类

在stock_common工程下定义公共分库算法类:

package com.itheima.stock.sharding;

import com.google.common.collect.Range;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.api.sharding.standard.RangeShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.RangeShardingValue;
import org.joda.time.DateTime;

import java.util.Collection;
import java.util.Date;
import java.util.Optional;
import java.util.stream.Collectors;

/**
 * @author by itheima
 * @Date 2022/6/13
 * @Description 定义公共的数据库分片算法类:包含精准匹配数据库和范围匹配数据库
 *  因为分库是根据日期分库的,一年一个库,所以片键的类型是Date
 */
public class CommonShardingAlgorithm4Db implements PreciseShardingAlgorithm<Date>, RangeShardingAlgorithm<Date> {
    
    

    /**
     * 精准匹配数据库的方法 cur_time 条件必须是 = 或者in
     * @param dsNames 所有可匹配数据源的集合 ds-2021 ds-2022
     * @param shardingValue
     * @return
     */
    @Override
    public String doSharding(Collection<String> dsNames, PreciseShardingValue<Date> shardingValue) {
    
    
        //1.思路:根据传入的日期值,获取年份字符串
        //获取分片字段的名称colume
//        String columnName = shardingValue.getColumnName();
        //获取逻辑表名称
//        String logicTableName = shardingValue.getLogicTableName();
        //获取分片值
        Date value = shardingValue.getValue();
        //获取年份字符串
        String year = new DateTime(value).getYear()+"";
        //2.获取数据源中以
        Optional<String> optional = dsNames.stream().filter(ds -> ds.endsWith(year)).findFirst();
        String actual=null;
        //判断是否有符合指定年份的数据源
        if (optional.isPresent()) {
    
    
            actual=optional.get();
        }
        return actual;
    }

    /**
     * 范围查询匹配数据源 关键字:between and
     * @param dsNames ds-2021 ds-2022
     * @param shardingValue
     * @return
     */
    @Override
    public Collection<String> doSharding(Collection<String> dsNames, RangeShardingValue<Date> shardingValue) {
    
    
        //获取分片字段名称
//        String columnName = shardingValue.getColumnName();
//        //获取逻辑表名称
//        String logicTableName = shardingValue.getLogicTableName();
        //1.获取范围封装对象
        Range<Date> valueRange = shardingValue.getValueRange();
        //2.1 判断是否有下限值
        if (valueRange.hasLowerBound()) {
    
    
            //获取下限日期
            Date lowerDate = valueRange.lowerEndpoint();
            //获取年份  dsNames--> ds_2021 ds_2022 ds_2023
            int year = new DateTime(lowerDate).getYear();//2022
            dsNames= dsNames.stream().filter(dsName->Integer.valueOf(dsName.substring(dsName.lastIndexOf("-")+1))>=year)
                    .collect(Collectors.toList());
        }
        //2.2 判断是否有上限值
        if (valueRange.hasUpperBound()) {
    
    
            Date upperDate = valueRange.upperEndpoint();
            int year = new DateTime(upperDate).getYear();
            dsNames= dsNames.stream().filter(dsName->Integer.valueOf(dsName.substring(dsName.lastIndexOf("-")+1))<=year)
                    .collect(Collectors.toList());
        }

        return dsNames;
    }
}

4.2 配置properties

在stock_backend工程下配置application-sharding.properties:

# 第二步:配置板块表的数据节点信息
spring.shardingsphere.sharding.tables.stock_block_rt_info.actual-data-nodes=ds-${2021..2022}.stock_block_rt_info
spring.shardingsphere.sharding.tables.stock_market_index_info.actual-data-nodes=ds-${2021..2022}.stock_market_index_info
spring.shardingsphere.sharding.tables.stock_outer_market_index_info.actual-data-nodes=ds-${2021..2022}.stock_outer_market_index_info

# 提取公共数据库分片算法配置类
common.algorithm4db=com.itheima.stock.sharding.CommonShardingAlgorithm4Db
common.algorithm4StockRtInfoTable=com.itheima.stock.sharding.ShardingAlgorithm4StockRtInfoTable

# 第三步:配置数据库的分片算法
# 分片列名称
spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.sharding-column=cur_time
# 精确分片算法类名称,用于 = 和 IN。该类需实现 PreciseShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.precise-algorithm-class-name=${common.algorithm4db}
# 范围分片算法类名称,用于 BETWEEN,可选。该类需实现 RangeShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.range-algorithm-class-name=${common.algorithm4db}

spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.sharding-column=cur_time
# 精确分片算法类名称,用于 = 和 IN。该类需实现 PreciseShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.precise-algorithm-class-name=${common.algorithm4db}
# 范围分片算法类名称,用于 BETWEEN,可选。该类需实现 RangeShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.range-algorithm-class-name=${common.algorithm4db}

spring.shardingsphere.sharding.tables.stock_outer_market_index_info.database-strategy.standard.sharding-column=cur_time
# 精确分片算法类名称,用于 = 和 IN。该类需实现 PreciseShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_outer_market_index_info.database-strategy.standard.precise-algorithm-class-name=${common.algorithm4db}
# 范围分片算法类名称,用于 BETWEEN,可选。该类需实现 RangeShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_outer_market_index_info.database-strategy.standard.range-algorithm-class-name=${common.algorithm4db}

4.3 功能测试

@Autowired
private StockBlockRtInfoMapper stockBlockRtInfoMapper;
@Test
public void testCommonSharding(){
    
    
  Date curDate= DateTime.parse("2022-01-03 09:30:00", DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")).toDate();
  List<StockBlockDomain> info = stockBlockRtInfoMapper.findBlockInfoByTimeLimit(curDate);
  System.out.println(info);
}

5、个股表分库分表配置

对于个股流水表来说分库策略与大盘板块一致,所以接下来,我们只定义好分表策略即可。

5.1 定义个股表公共分表策略

在stock_common工程下将精准和范围匹配表的接口实现合并到一个算法类下:

package com.itheima.stock.sharding;

import com.google.common.collect.Range;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.api.sharding.standard.RangeShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.RangeShardingValue;
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;

import java.util.Collection;
import java.util.Date;
import java.util.Optional;
import java.util.stream.Collectors;

/**
 * @author by itheima
 * @Date 2022/6/13
 * @Description 定义股票流水表的分片算法类:包含精准匹配表和范围匹配表
 *  因为分库是根据日期分库的,一年一个库,一个月一张表,也就是说每个库内都包含12张表,所以片键的类型是Date
 */
public class ShardingAlgorithm4StockRtInfoTable implements PreciseShardingAlgorithm<Date>, RangeShardingAlgorithm<Date> {
    
    

    /**
     * 精准匹配表的方法 cur_time 条件必须是 = 或者in
     * @param tbNames 所有可匹配表的集合 stock_rt_info_202101....stock_rt_info_202112
     *                                stock_rt_info_202201....stock_rt_info_202212
     * @param shardingValue
     * @return
     */
    @Override
    public String doSharding(Collection<String> tbNames, PreciseShardingValue<Date> shardingValue) {
    
    
        //1.思路:根据传入的日期值,获取年份字符串
        //获取分片字段的名称colume
//        String columnName = shardingValue.getColumnName();
        //获取逻辑表名称
//        String logicTableName = shardingValue.getLogicTableName();
        //获取分片值
        Date value = shardingValue.getValue();
        //获取年月组成的字符串
        String yearMonth = new DateTime(value).toString(DateTimeFormat.forPattern("yyyyMM"));
        //过滤表的名称集合,获取名称后缀与yearMonth一致的表名称
        Optional<String> optional = tbNames.stream().filter(tbName -> tbName.endsWith(yearMonth)).findFirst();
        String tbName=null;
        if (optional.isPresent()) {
    
    
            tbName=optional.get();
        }
        return tbName;
    }

    /**
     * 范围查询匹配表 关键字:between and
     * @param tbNames 所有可匹配表的集合 stock_rt_info_202101....stock_rt_info_202112
     *                                stock_rt_info_202201....stock_rt_info_202212
     * @param shardingValue
     * @return
     */
    @Override
    public Collection<String> doSharding(Collection<String> tbNames, RangeShardingValue<Date> shardingValue) {
    
    
        //获取分片字段名称
//        String columnName = shardingValue.getColumnName();
//        //获取逻辑表名称
//        String logicTableName = shardingValue.getLogicTableName();
        //1.获取范围封装对象
        Range<Date> valueRange = shardingValue.getValueRange();
        //2.1 判断是否有下限值
        if (valueRange.hasLowerBound()) {
    
    
            //获取下限日期
            Date lowerDate = valueRange.lowerEndpoint();
            //获取年份  dsNames--> ds_2021 ds_2022 ds_2023
            //获取年月组成的字符串
            String yearMonth = new DateTime(lowerDate).toString(DateTimeFormat.forPattern("yyyyMM"));
            Integer yearM = Integer.valueOf(yearMonth);
            tbNames= tbNames.stream().filter(tbName->Integer.valueOf(tbName.substring(tbName.lastIndexOf("_")+1))>=yearM)
                    .collect(Collectors.toList());
        }
        //2.2 判断是否有上限值
        if (valueRange.hasUpperBound()) {
    
    
            Date upperDate = valueRange.upperEndpoint();
            String yearMonth = new DateTime(upperDate).toString(DateTimeFormat.forPattern("yyyyMM"));
            Integer yearM = Integer.valueOf(yearMonth);
            tbNames= tbNames.stream().filter(tbName->Integer.valueOf(tbName.substring(tbName.lastIndexOf("_")+1))<=yearM)
                    .collect(Collectors.toList());
        }
        return tbNames;
    }
}

5.2 配置个股分库分表

在stock_backend工程下配置分库分表策略:

# 配置股票流水节点信息
spring.shardingsphere.sharding.tables.stock_rt_info.actual-data-nodes=ds-2021.stock_rt_info_${202101..202112},ds-2022.stock_rt_info_${202201..202212}
# 抽取公共配置类变量
common.algorithm4StockRtInfoTable=com.itheima.stock.sharding.ShardingAlgorithm4StockRtInfoTable
# 配置股票流水库分片策略
spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.sharding-column=cur_time
# 精确分片算法类名称,用于 = 和 IN。该类需实现 PreciseShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.precise-algorithm-class-name=${common.algorithm4db}
# 范围分片算法类名称,用于 BETWEEN,可选。该类需实现 RangeShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.range-algorithm-class-name=${common.algorithm4db}

# 第四步:配置表的分片算法
# 因为stock_block_rt_info板块表仅仅按照年分库,并没有库内分片的操作,也就是说每个库内的表名称都一样,且只有一个,所以不需要定义分表的算法类
# 配置股票流水表的分片算法
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.sharding-column=cur_time
# 精确分片算法类名称,用于 = 和 IN。该类需实现 PreciseShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.precise-algorithm-class-name=${common.algorithm4StockRtInfoTable}
# 范围分片算法类名称,用于 BETWEEN,可选。该类需实现 RangeShardingAlgorithm 接口并提供无参数的构造器
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.range-algorithm-class-name=${common.algorithm4StockRtInfoTable}

5.3测试

    @Autowired
    private StockRtInfoMapper stockRtInfoMapper;

    /**
     * @Description 测试分库分表算法类
     */
    @Test
    public void testShardingDbAndTb(){
    
    
        //截止时间
        Date endTime=DateTime.parse("2022-05-22 09:30:00",DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")).toDate();
        //开始时间
        Date startTime=DateTime.parse("2021-01-01 09:30:00",DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")).toDate();
        //根据指定日期范围查询周K线数据
        List<Stock4EvrWeekDomain> infos=stockRtInfoMapper.getHalfWeekLineData("000017",startTime,endTime);
        System.out.println(infos);
    }

最终application-shrding.properties配置:

# 1. 配置数据源


spring.shardingsphere.datasource.names=defultdb,ds-2021,ds-2022,ds-2023,ds-2024


spring.shardingsphere.datasource.defultdb.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.defultdb.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.defultdb.url=jdbc:mysql://127.0.0.1:3306/stock_sys_db?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
spring.shardingsphere.datasource.defultdb.username=root
spring.shardingsphere.datasource.defultdb.password=root



spring.shardingsphere.datasource.ds-2021.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2021.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2021.url=jdbc:mysql://127.0.0.1:3306/stock_db_2021?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
spring.shardingsphere.datasource.ds-2021.username=root
spring.shardingsphere.datasource.ds-2021.password=root



spring.shardingsphere.datasource.ds-2022.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2022.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2022.url=jdbc:mysql://127.0.0.1:3306/stock_db_2022?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
spring.shardingsphere.datasource.ds-2022.username=root
spring.shardingsphere.datasource.ds-2022.password=root



spring.shardingsphere.datasource.ds-2023.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2023.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2023.url=jdbc:mysql://127.0.0.1:3306/stock_db_2023?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
spring.shardingsphere.datasource.ds-2023.username=root
spring.shardingsphere.datasource.ds-2023.password=root



spring.shardingsphere.datasource.ds-2024.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2024.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2024.url=jdbc:mysql://127.0.0.1:3306/stock_db_2024?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=Asia/Shanghai
spring.shardingsphere.datasource.ds-2024.username=root
spring.shardingsphere.datasource.ds-2024.password=root


# 2. 配置逻辑表数据节点
spring.shardingsphere.sharding.tables.stock_block_rt_info.actual-data-nodes=ds-${2021..2024}.stock_block_rt_info
spring.shardingsphere.sharding.tables.stock_outer_market_index.actual-data-nodes=ds-${2021..2024}.stock_outer_market_index
spring.shardingsphere.sharding.tables.stock_market_index_info.actual-data-nodes=ds-${2021..2024}.stock_market_index_info
spring.shardingsphere.sharding.tables.stock_rt_info.actual-data-nodes=ds-2021.stock_rt_info_${202101..202112},ds-2022.stock_rt_info_${202201..202212},ds-2023.stock_rt_info_${202301..202312},ds-2024.stock_rt_info_${202301..202312}


# 3. 配置分库策略
common.sharding4db=com.jixu.stock.sharding.Alg4DB
common.sharding4tb=com.jixu.stock.sharding.Alg4TB

spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.sharding-column= cur_time
spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.precise-algorithm-class-name=${common.sharding4db}
spring.shardingsphere.sharding.tables.stock_block_rt_info.database-strategy.standard.range-algorithm-class-name=${common.sharding4db}

spring.shardingsphere.sharding.tables.stock_outer_market_index.database-strategy.standard.sharding-column= cur_time
spring.shardingsphere.sharding.tables.stock_outer_market_index.database-strategy.standard.precise-algorithm-class-name=${common.sharding4db}
spring.shardingsphere.sharding.tables.stock_outer_market_index.database-strategy.standard.range-algorithm-class-name=${common.sharding4db}

spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.sharding-column= cur_time
spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.precise-algorithm-class-name=${common.sharding4db}
spring.shardingsphere.sharding.tables.stock_market_index_info.database-strategy.standard.range-algorithm-class-name=${common.sharding4db}

spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.sharding-column= cur_time
spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.precise-algorithm-class-name=${common.sharding4db}
spring.shardingsphere.sharding.tables.stock_rt_info.database-strategy.standard.range-algorithm-class-name=${common.sharding4db}


# 4. 配置分表策略
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.sharding-column= cur_time
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.precise-algorithm-class-name=${common.sharding4tb}
spring.shardingsphere.sharding.tables.stock_rt_info.table-strategy.standard.range-algorithm-class-name=${common.sharding4tb}


# 5. 其他配置

# 5.1 配置默认数据源
spring.shardingsphere.sharding.default-data-source-name=defultdb

# 5.2 配置显示sql
spring.shardingsphere.props.sql.show=true

# 5.3 配置广播表
spring.shardingsphere.sharding.broadcast-tables=stock_business

6、分库分表注意事项

基于sharding-jdbc实践分库分表注意事项:

  • 条件查询时分片字段不要使用函数处理,否则分片算法失效,导致全节点查询
    • 举例:select * from stock_rt_info where date_format(cur_time,‘%Y%m%d’)=‘20220910’ ,函数会造成sharding的分片失效,导致全节点查询;
    • 同时在索引角度看,如果查询的分片字段使用函数,会导致索引失效,导致查询性能较低;
  • 条件查询时尽量使用符合sharding分片条件的关键字
    • 精准查询尽量使用in =,而范围查询尽量使用between ;
  • sharding-jdbc对嵌套查询处理不友好
    • 如果嵌套查询的话,那么最好子查询的条件只命中单张表。如果子查询的条件关联了多张表,那么交易分步骤拆分实现;
    • 示例:我们项目中的K线统计中,需要将SQL拆分,然后分步骤实现;

猜你喜欢

转载自blog.csdn.net/xiugtt6141121/article/details/143019781