当当分库分表(一)

1、概述

当单表的记录条数达到千万级别后,就要考虑使用分库分表技术框架了。本文介绍当当分库分表技术框架Sharding-JDBC,主页http://shardingjdbc.io。下载1.5.4.1源码,https://github.com/shardingjdbc/sharding-jdbc/releases。

2、开发环境

win7、JDK1.7、IntelliJ IDEA 2017.2.5。

3、sharding-jdbc-example-mybatis

File-Open...,选择sharding-jdbc-1.5.4.1下的sharding-jdbc-example,先从sharding-jdbc-example-mybatis项目开始,如图01。

3.1 准备工作

修改shardingContext.xml里访问MySQL的用户名和密码,我的密码是:123456。

<bean id="ds_0" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost:3306/ds_0"/>
    <property name="username" value="root"/>
    <property name="password" value="123456"/>
</bean>

<bean id="ds_1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost:3306/ds_1"/>
    <property name="username" value="root"/>
    <property name="password" value="123456"/>
</bean>

根据mysql.sql,在ds_0和ds_1里分别建表:t_order_0、t_order_1、t_order_item_0、t_order_item_1。本例不涉及t_order_item_0、t_order_item_1。

要认真看看shardingContext.xml里的内容,并和传统Web应用里的数据源配置做下对比。通俗地讲,id为"transactionManager"的bean引用了数据源"shardingDataSource",而后者根据某种算法,依据user_id的值选择真正的数据源"ds_0"或"ds_1"。之后,再根据某种算法,依据order_id的值选择插入到t_order_0或t_order_1。

上面提到的两种算法就是shardingContext.xml里的SingleKeyModuloDatabaseShardingAlgorithm.java(以下简称01类)和SingleKeyModuloTableShardingAlgorithm.java(以下简称02类)。对上述两类中的三个核心方法添加方法参数打印语句,先感性认识再理性理解!

public final class SingleKeyModuloDatabaseShardingAlgorithm implements SingleKeyDatabaseShardingAlgorithm<Integer> {
    
    @Override
    public String doEqualSharding(final Collection<String> availableTargetNames, final ShardingValue<Integer> shardingValue) {
        System.out.println("###### 01 doEqualSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        for (String each : availableTargetNames) {
            if (each.endsWith(shardingValue.getValue() % 2 + "")) {
                System.out.println("###### 01 doEqualSharding(),命中=" + each);
                return each;
            }
        }
        throw new UnsupportedOperationException();
    }
    
    @Override
    public Collection<String> doInSharding(final Collection<String> availableTargetNames, final ShardingValue<Integer> shardingValue) {
        System.out.println("###### 01 doInSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        Collection<String> result = new LinkedHashSet<>(availableTargetNames.size());
        Collection<Integer> values = shardingValue.getValues();
        for (Integer value : values) {
            for (String each : availableTargetNames) {
                if (each.endsWith(value % 2 + "")) {
                    result.add(each);
                }
            }
        }
        System.out.println("###### 01 doInSharding(),命中=" + result);
        return result;
    }
    
    @Override
    public Collection<String> doBetweenSharding(final Collection<String> availableTargetNames, final ShardingValue<Integer> shardingValue) {
        System.out.println("###### 01 doBetweenSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        Collection<String> result = new LinkedHashSet<>(availableTargetNames.size());
        Range<Integer> range = shardingValue.getValueRange();
        for (Integer value = range.lowerEndpoint(); value <= range.upperEndpoint(); value++) {
            for (String each : availableTargetNames) {
                if (each.endsWith(value % 2 + "")) {
                    result.add(each);
                }
            }
        }
        System.out.println("###### 01 doBetweenSharding(),命中=" + result);
        return result;
    }
}

public final class SingleKeyModuloTableShardingAlgorithm implements SingleKeyTableShardingAlgorithm<Long> {
    
    @Override
    public String doEqualSharding(final Collection<String> availableTargetNames, final ShardingValue<Long> shardingValue) {
        System.out.println("###### 02 doEqualSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        for (String each : availableTargetNames) {
            if (each.endsWith(shardingValue.getValue() % 2 + "")) {
                System.out.println("###### 02 doEqualSharding(),命中=" + each);
                return each;
            }
        }
        throw new UnsupportedOperationException();
    }
    
    @Override
    public Collection<String> doInSharding(final Collection<String> availableTargetNames, final ShardingValue<Long> shardingValue) {
        System.out.println("###### 02 doInSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        Collection<String> result = new LinkedHashSet<>(availableTargetNames.size());
        Collection<Long> values = shardingValue.getValues();
        for (Long value : values) {
            for (String each : availableTargetNames) {
                if (each.endsWith(value % 2 + "")) {
                    result.add(each);
                }
            }
        }
        System.out.println("###### 02 doInSharding(),命中=" + result);
        return result;
    }
    
    @Override
    public Collection<String> doBetweenSharding(final Collection<String> availableTargetNames, final ShardingValue<Long> shardingValue) {
        System.out.println("###### 02 doBetweenSharding(),availableTargetNames=" + availableTargetNames + ",shardingValue=" + shardingValue);
        Collection<String> result = new LinkedHashSet<>(availableTargetNames.size());
        Range<Long> range = shardingValue.getValueRange();
        for (Long i = range.lowerEndpoint(); i <= range.upperEndpoint(); i++) {
            for (String each : availableTargetNames) {
                if (each.endsWith(i % 2 + "")) {
                    result.add(each);
                }
            }
        }
        System.out.println("###### 02 doBetweenSharding(),命中=" + result);
        return result;
    }
}

3.2 执行Main类中的main()方法

分析Dao操作与对应的日志输出。


orderService.clear();不涉及上述两个算法。


orderService.fooService();

###### 01 doEqualSharding(),availableTargetNames=[ds_0, ds_1],shardingValue=ShardingValue(logicTableName=t_order, columnName=user_id, value=10, values=[], valueRange=null)
###### 01 doEqualSharding(),命中=ds_0
###### 02 doEqualSharding(),availableTargetNames=[t_order_1, t_order_0],shardingValue=ShardingValue(logicTableName=t_order, columnName=order_id, value=149948054085042176, values=[], valueRange=null)
###### 02 doEqualSharding(),命中=t_order_0
Generated key1 of order_id:149948054085042176
###### 01 doEqualSharding(),availableTargetNames=[ds_0, ds_1],shardingValue=ShardingValue(logicTableName=t_order, columnName=user_id, value=11, values=[], valueRange=null)
###### 01 doEqualSharding(),命中=ds_1
###### 02 doEqualSharding(),availableTargetNames=[t_order_1, t_order_0],shardingValue=ShardingValue(logicTableName=t_order, columnName=order_id, value=149948054353477632, values=[], valueRange=null)
###### 02 doEqualSharding(),命中=t_order_0
Generated key2 of order_id:149948054353477632
###### 01 doInSharding(),availableTargetNames=[ds_0, ds_1],shardingValue=ShardingValue(logicTableName=t_order, columnName=user_id, value=null, values=[10, 11], valueRange=null)
###### 01 doInSharding(),命中=[ds_0, ds_1]

通俗地讲,当执行INSERT INTO t_order (user_id, status) VALUES...时:
1、根据01类的doEqualSharding()方法,命中了数据源ds_0。其中,可供选择的数据源名称为:ds_0, ds_1。逻辑表名为t_order。分库列为user_id,其值为10。
2、根据02类的doEqualSharding()方法,命中了表t_order_0。其中,可供选择的表名称为t_order_1, t_order_0。逻辑表名为t_order。分表列为order_id,其值为149948054085042176。

第二个INSERT与第一个类似。

当执行UPDATE t_order SET status = 'UPDATED' WHERE user_id in(...)时:
1、根据01类的doInSharding()方法(注意:不是doEqualSharding()方法了),命中了数据源ds_0, ds_1。其中,可供选择的数据源名称为:ds_0, ds_1。逻辑表名为t_order。分片列为user_id,其值的集合为[10, 11]。
2、因为不涉及分片列为oder_id,未使用到02类。

orderService.select();不涉及上述两个算法。

orderService.fooServiceWithFailure();等同于orderService.fooService();

逻辑表,是相对于物理表而言的。因为Sharding-JDBC的存在,站在MyBatis映射文件的角度来说,它根本不知道有ds_0、ds_1、t_order_0、t_order_1等的存在,只知道t_order,仿佛在操作单数据源似的。

3.3 再回头看看shardingContext.xml


猜你喜欢

转载自blog.csdn.net/jinjiankang/article/details/78846518
今日推荐