Sharding Sphere简介

5.0版本官网 https://shardingsphere.apache.org/document/5.0.0/cn/overview/#%E7%AE%80%E4%BB%8B

项目源码地址 https://gitee.com/nssnail/middleware-/tree/master/sharding-jdbc

Apache ShardingSphere 产品定位为 Database Plus，旨在构建多模数据库上层的标准和生态。 它关注如何充分合理地利用数据库的计算和存储能力，而并非实现一个全新的数据库。ShardingSphere 站在数据库的上层视角，关注他们之间的协作多于数据库自身。

连接、增量和可插拔是 Apache ShardingSphere 的核心概念。

• 连接：通过对数据库协议、SQL 方言以及数据库存储的灵活适配，快速的连接应用与多模式的异构数据库；
• 增量：获取数据库的访问流量，并提供流量重定向（数据分片、读写分离、影子库）、流量变形（数据加密、数据脱敏）、流量鉴权（安全、审计、权限）、流量治理（熔断、限流）以及流量分析（服务质量分析、可观察性）等透明化增量功能；
• 可插拔：项目采用微内核 + 三层可插拔模型，使内核、功能组件以及生态对接完全能够灵活的方式进行插拔式扩展，开发者能够像使用积木一样定制属于自己的独特系统。

ShardingSphere 已于2020年4月16日成为 Apache 软件基金会的顶级项目。

sharding-JDBC最早是当当网内部使用的一款分库分表框架，到2017年的时候才开始对外开源，这几年在大量社区贡献者的不断迭代下，功能也逐渐完善，现已更名为shardingsphere，2020年4月16日正式成为Apache软件基金会的顶级项目。

ShardingSphere生态包含三款开源分布式数据库中间件解决方案，Sharding-JDBC、Sharding-Proxy、Sharding Sidecar。

Apache ShardingSphere 5.x 版本开始致力于提供可插拔架构，项目的功能组件能够灵活的以可插拔的方式进行扩展。 目前，数据分片、读写分离、数据加密、影子库压测等功能，以及对 MySQL、PostgreSQL、SQLServer、Oracle 等 SQL 与协议的支持，均通过插件的方式织入项目。 开发者能够像使用积木一样定制属于自己的独特系统。Apache ShardingSphere 目前已提供数十个 SPI 作为系统的扩展点，而且仍在不断增加中。

如下图，是Sharding-Sphere的整体架构。

Sharding-JDBC、Sharding-Proxy、Sharding Sidecar各个功能对比如下

开始使用Sharding JDBC

数据分片

简单示例

添加两个数据库，shard01和shard02

创建maven项目，并添加pom.xml依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.2</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.sise</groupId>
    <artifactId>sharding-jdbc-example</artifactId>
    <version>1.0-SNAPSHOT</version>


    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>shardingsphere-jdbc-core</artifactId>
            <version>5.0.0-alpha</version>
        </dependency>
        <dependency>
            <groupId>com.zaxxer</groupId>
            <artifactId>HikariCP</artifactId>
            <version>3.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

添加order实体类

@Data
public class Order implements Serializable {

    private static final long serialVersionUID = 6550464567497951903L;

    private long orderId;

    private int userId;

    private String status;

    private Date createDate;
}

添加OrderRepository接口，数据层操作

public interface OrderRepository {

    /**
     * 创建sql
     * @throws SQLException
     */
    void createTableIfNotExists() throws SQLException;

    /**
     * 插入订单
     * @param order
     * @return
     * @throws SQLException
     */
    Long insert(final Order order) throws SQLException;
}

OrderRepository实现类

public class OrderRepositoryImpl implements OrderRepository {

    private final DataSource dataSource;

    public OrderRepositoryImpl(final DataSource dataSource) {
        this.dataSource = dataSource;
    }
    @Override
    public void createTableIfNotExists() throws SQLException {
        String sql = "CREATE TABLE IF NOT EXISTS t_order (order_id BIGINT NOT NULL AUTO_INCREMENT, user_id INT NOT NULL, status VARCHAR(50),create_date date,PRIMARY KEY (order_id))";
        try (Connection connection = dataSource.getConnection();
             Statement statement = connection.createStatement()) {
            statement.executeUpdate(sql);
        }
    }
    @Override
    public Long insert(final Order order) throws SQLException {
        String sql = "INSERT INTO t_order (user_id, status,create_date) VALUES ( ?, ?, ?)";
        try (Connection connection = dataSource.getConnection();
            PreparedStatement preparedStatement = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) {
            preparedStatement.setInt(1, order.getUserId());
            preparedStatement.setString(2, order.getStatus());
            preparedStatement.setDate(3, new Date(order.getCreateDate().getTime()));
            preparedStatement.executeUpdate();
            try (ResultSet resultSet = preparedStatement.getGeneratedKeys()) {
                if (resultSet.next()) {
                    order.setOrderId(resultSet.getLong(1));
                }
            }
        }
        return order.getOrderId();
    }
}

创建OrderService接口，用于业务层操作

public interface OrderService {
    /**
     * 初始化表结构
     *
     * @throws SQLException SQL exception
     */
    void initEnvironment() throws SQLException;

    /**
     * 执行成功
     *
     * @throws SQLException SQL exception
     */
    void processSuccess() throws SQLException;

}

OrderService实现类

public class OrderServiceImpl implements OrderService {
    private final OrderRepository orderRepository;
    Random random=new Random();

    public OrderServiceImpl(final DataSource dataSource) {
        orderRepository=new OrderRepositoryImpl(dataSource);
    }

    @Override
    public void initEnvironment() throws SQLException {
        orderRepository.createTableIfNotExists();
    }

    @Override
    public void processSuccess() throws SQLException {
        System.out.println("-------------- Process Success Begin ---------------");
        List<Long> orderIds = insertData();
        System.out.println("-------------- Process Success Finish --------------");
    }
    private List<Long> insertData() throws SQLException {
        System.out.println("---------------------------- Insert Data ----------------------------");
        List<Long> result = new ArrayList<>(10);
        for (int i = 1; i <= 10; i++) {
            Order order = insertOrder();
            result.add(order.getOrderId());
        }
        return result;
    }
    private Order insertOrder() throws SQLException {
        Order order = new Order();
        order.setUserId(random.nextInt(10000));
        order.setStatus("INSERT_TEST");
        order.setCreateDate(new Date());
        orderRepository.insert(order);
        return order;
    }
}

创建数据源工具类

public class DataSourceUtil {

    private static final String HOST = "localhost";

    private static final int PORT = 3306;

    private static final String USER_NAME = "root";

    private static final String PASSWORD = "root";

    public static DataSource createDataSource(final String dataSourceName) {
        HikariDataSource result = new HikariDataSource();
        result.setDriverClassName("com.mysql.jdbc.Driver");
        result.setJdbcUrl(String.format("jdbc:mysql://%s:%s/%s?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8", HOST, PORT, dataSourceName));
        result.setUsername(USER_NAME);
        result.setPassword(PASSWORD);
        return result;
    }
}

以上都是为测试做准备

创建sharding工具类(重点)

public class ShardingDatabaseAndTableConfiguration {

    private static Map<String, DataSource> createDataSourceMap(){
        //代表真实的数据源
        Map<String,DataSource> dataSourceMap=new HashMap<>();
        dataSourceMap.put("ds0",DataSourceUtil.createDataSource("shard01"));
        dataSourceMap.put("ds1",DataSourceUtil.createDataSource("shard02"));
        return dataSourceMap;
    }
    //创建分片规则
    // * 针对数据库
    // * 针对表
    //* 一定要配置分片键
    //* 一定要配置分片算法
    //* 完全唯一id的问题
    private static ShardingRuleConfiguration createShardingRuleConfiguration(){
        ShardingRuleConfiguration configuration=new ShardingRuleConfiguration();
        //把逻辑表和真实表的对应关系添加到分片规则配置中
        configuration.getTables().add(getOrderTableRuleConfiguration());
        //设置数据库分库规则
        configuration.setDefaultDatabaseShardingStrategy(
                new StandardShardingStrategyConfiguration
                        ("user_id","db-inline"));
        Properties properties=new Properties();
        properties.setProperty("algorithm-expression","ds${user_id%2}");
        //设置分库策略
        configuration.getShardingAlgorithms().
                put("db-inline",new ShardingSphereAlgorithmConfiguration("INLINE",properties));

        //设置表的分片规则（数据的水平拆分）
        configuration.setDefaultTableShardingStrategy(new StandardShardingStrategyConfiguration
                ("order_id","order-inline"));
        //设置分表策略
        Properties props=new Properties();
        props.setProperty("algorithm-expression","t_order_${order_id%2}");
        configuration.getShardingAlgorithms().put("order-inline",
                new ShardingSphereAlgorithmConfiguration("INLINE",props));
        //设置主键生成策略
        // * UUID
        // * 雪花算法
        Properties idProperties=new Properties();
        idProperties.setProperty("worker-id","123");
        configuration.getKeyGenerators().put("snowflake",new ShardingSphereAlgorithmConfiguration(
                "SNOWFLAKE",idProperties));
        return configuration;
    }
    //配置逻辑表以及表的id策略
    private static ShardingTableRuleConfiguration getOrderTableRuleConfiguration(){
        ShardingTableRuleConfiguration tableRuleConfiguration=
                new ShardingTableRuleConfiguration("t_order","ds${0..1}.t_order_${0..1}");
        tableRuleConfiguration.setKeyGenerateStrategy(new KeyGenerateStrategyConfiguration("order_id","snowflake"));
        return tableRuleConfiguration;
    }

    public static DataSource getDatasource() throws SQLException {
        return ShardingSphereDataSourceFactory
                .createDataSource(createDataSourceMap(), Collections.singleton(createShardingRuleConfiguration()),new Properties());
    }
}

代码解读

创建数据源集合，用于分库，单库put一个即可

private static Map<String, DataSource> createDataSourceMap(){
    //代表真实的数据源
    Map<String,DataSource> dataSourceMap=new HashMap<>();
    dataSourceMap.put("ds0",DataSourceUtil.createDataSource("shard01"));
    dataSourceMap.put("ds1",DataSourceUtil.createDataSource("shard02"));
    return dataSourceMap;
}

配置逻辑表以及表的id策略

sharding jdbc会在表达式中去寻找真实表

ds${0..1}.t_order_${0..1}为Groovy表达式

最终会被解析成

shard01.t_order_0、shard01.t_order_1、shard02.t_order_0、shard02.t_order_1

由此可见这是个排列组合

setKeyGenerateStrategy设置order_id为雪花算法策略，具体配置在以下配置中,SNOWFLAKE为雪花算法

可以使用雪花算法或者UUID算法，具体文档如下

ID算法文档

ShardingRuleConfiguration configuration=new ShardingRuleConfiguration();

//把逻辑表和真实表的对应关系添加到分片规则配置中

configuration.getTables().add(getOrderTableRuleConfiguration());

..<