The FISCO BCOS team conducted extensive discussions with enthusiastic community partners and industry experts on distributed storage. Here, I would like to share with you my exchange experience, which may help you better understand and use distributed storage:

The distributed storage of FISCO BCOS 2.0 adopts the library table style, and the CRUD operation conforms to business habits.

Without the contract storage variable mode, the embedded coupling between the contract and data is deconstructed, and the contract upgrade is easier.

The storage access engine logic and data structure are more intuitive, easy to adapt to various storage engines, and have large expansion space.

The data itself is stored in a determinant, without the intertwined relationship of the MPT tree, and it is easier to take snapshots and cut and migrate.

The structure index data of the table plus the primary key has high access efficiency and easier concurrent access.

The storage overhead is less, the capacity model is linearly related to the number of transactions and the number of states, and it is easier to predict the business capacity, which is very meaningful for massive services.

In terms of details, the state MPT is weakened, but the transaction and receipt MPT are retained. It can still support light clients, adopt process proof and existence proof, and do not rely on volatile state, and will not affect the realization of cross-chain.

The state is checked by incremental HASH, and the state set generated by each block transaction will be strictly checked by the whole network to ensure consistency.

It was initially built for SQL types, supporting engines such as MySQL and Oracle, and then adapted to NoSQL types, such as LevelDB. More high-speed and massive storage engines will be adapted in the future, and the optimal solution will be explored in the triangular relationship of [single IO delay/concurrency efficiency/capacity expansion].

Although distributed storage is a big project (a few fast gunners in the team dared to take it out to see people after a year of work), it is very simple to use. This article will talk about the experience process of distributed storage. For initial contact with users, it is recommended to start with the previous article: "Distributed Storage Architecture Design" (click to browse).

Configure distributed storage

Distributed storage supports multiple storage engines, which can be flexibly selected according to business needs and deployment environments, and can be configured as different storage engines.

The basic data such as blocks and transactions of the blockchain are stored in a database table structure, and the storage method of state data can be configured as a database table structure or MPT to meet the needs of different scenarios.

The configuration items of distributed storage are located in the configuration file of the group. Each group can use a separate storage strategy. The group configuration file is located in the blockchain node under the path named conf/group.[group number].genesis , such as group.1.genesis, once the group is started, the related configuration of the group's distributed storage cannot be changed.

Examples of distributed storage configuration items are as follows:

[storage]

type=LevelDB: DB engine type for distributed storage, supporting "LevelDB" and "External" (rc2 version)

[state]

type=storage: state type, currently supports storage state and MPT state, the default is storage state

It is recommended to use storage state , unless MPT must be used to trace the global historical state, MPT State is not recommended.

Developed using CRUD smart contracts

Distributed storage provides a dedicated CRUD interface that supports contracts to directly access the underlying storage table.

To access CRUD, you need to refer to the Table.sol interface of the smart contract dedicated to distributed storage. This interface is a database contract, which can create tables, and perform addition, deletion, modification and query operations on tables.

Reference Table.sol

import "./Table.sol";

The interface of Table.sol includes:

createTable // create a table
select(string, Condition) //query data
insert(string, Entry) //insert data
update(string, Entry, Condition) //update data
remove(string, Condition) //delete data

The usage of each interface is as follows:

create table

// TableFactory的地址固定为0x1001
TableFactory tf = TableFactory(0x1001); 

// 创建t_test表，表的key_field为name，value_field为item_id,item_name 
// key_field表示分布式存储主key value_field表示表中的列，可以有多列，以逗号分隔
int count = tf.createTable("t_test", "name", "item_id,item_name");

Query data

TableFactory tf = TableFactory(0x1001);
Table table = tf.openTable("t_test");

// 条件为空表示不筛选 也可以根据需要使用条件筛选
Condition condition = table.newCondition();

Entries entries = table.select(name, condition);

insert data

TableFactory tf = TableFactory(0x1001);
Table table = tf.openTable("t_test");

Entry entry = table.newEntry();
entry.set("name", name);
entry.set("item_id", item_id);
entry.set("item_name", item_name);

int count = table.insert(name, entry);

update data

TableFactory tf = TableFactory(0x1001);
Table table = tf.openTable("t_test");

Entry entry = table.newEntry();
entry.set("item_name", item_name);
 
Condition condition = table.newCondition();
condition.EQ("name", name);
condition.EQ("item_id", item_id);

int count = table.update(name, entry, condition);

delete data

TableFactory tf = TableFactory(0x1001);
Table table = tf.openTable("t_test");

Condition condition = table.newCondition();
condition.EQ("name", name);
condition.EQ("item_id", item_id);

int count = table.remove(name, condition);

PS/

The optimization of storage architecture is a basic project, but also a big project. The realized transformation is actually an evolution of the architectural world view, and the impact will be more far-reaching than the functional points seen. What these two articles describe is only the tip of the iceberg of distributed storage. For more principles and use cases, please refer to:

https://fisco-bcos-documentation.readthedocs.io/zh_CN/release-2.0/docs/manual/smart_contract.html

We encourage community partners such as institutional members and developers to participate in open source co-construction. It will be even more remarkable if you are with us. Various ways to participate:

1 Enter the WeChat community, and chat with the most active and top-notch team in the circle anytime, anywhere on technical topics (to join the group, please add the assistant WeChat, WeChat ID: fiscobcosfan);

2 Subscribe to our official account: "FISCO BCOS Open Source Community", we have prepared a development database, the latest FISCO BCOS news, events, competitions and other information for you;

3 Come to Meetup to communicate face-to-face with the development team. FISCO BCOS is holding touring Meetups across the country, Shenzhen, Beijing, Shanghai, Chengdu... Welcome to your official account to find the nearby Meetup in the menu bar [Find Events], go to meet technical experts, and enjoy Talk about hard core technology;

4 Participate in code contribution. You can submit an Issue on Github for problem communication. You are welcome to submit a Pull Request to FISCO BCOS, including but not limited to document modification, bug fixes found, and submission of new features.

Code Contribution Guidelines:

https://github.com/FISCO-BCOS/FISCO-BCOS/blob/master/docs/CONTRIBUTING_CN.md

To learn more about dry goods, please pay attention to the FISCO BCOS open source community public account, and visit the FISCO BCOS code warehouse to download all source codes of the project: https://github.com/FISCO-BCOS/FISCO-BCOS , welcome to click the star collection in the upper right corner of the page , to get the latest version.

FISCO BCOS 2.0 tutorial: Distributed storage experience

Configure distributed storage

Developed using CRUD smart contracts

Guess you like