Blockchain 2.0 Notes

Blockchain 2.0

Ethereum Overview

Several improvements over Bitcoin

1. Shorten the block generation time to more than 10 seconds
2. ghost consensus mechanism
3. mining puzzle
   • BTC: Computationally intensive
   • ETH: memory-hard (limited ASIC)
4. proof of work->proof of stake
5. Support for smart contracts
   • BTC：decentralized currency
   • ETH: decentralized contract (decentralized contract) advantages

Ethereum account

Bitcoin is a transaction-based ledger, and each transaction is composed of inputs and outputs. The input references the output of the previous transaction to prove that the sender has access to those Bitcoins. The output specifies the recipient address and the number of Bitcoins transferred to the recipient. Therefore, if you want to know the balance, you have to go to UTXO to find the associated account.

for example:

Suppose Alice owns 10 Bitcoins and wants to send 3 of them to Bob. This will produce a transaction whose input is 10 Bitcoins (from Alice's previous transaction output) and the outputs are 3 Bitcoins (to Bob) and 7 Bitcoins (returned to Alice herself, since the remaining Bitcoins must returned to the sender).

When this transaction is confirmed, the previous concept of "account balance" of 10 Bitcoins no longer exists in the system. Instead, there will be two unspent transaction outputs in the ledger: one for Bob’s 3 Bitcoins and one for Alice’s 7 Bitcoins. These unspent transaction outputs can be used to build the next transaction.

The Ethereum system is an account-based ledger.

In the account-based ledger model, the blockchain records each account and its corresponding balance. Each account has an address (similar to a bank account number) and an Ether balance associated with it. When a transaction occurs on the Ethereum network, it involves the transfer of ether between different accounts.

Suppose account B wants to send ether to account C. A transaction is created, specifying the sender (account B), the receiver (account C), and the amount of ether to be transferred. After a transaction is confirmed and added to a block in the Ethereum blockchain, the account balance is updated accordingly. Ether is deducted from account B's balance, while the same amount is added to account C's balance.

Advantages of account based model

1. In line with people’s subjective feelings, it is similar to current bank transactions.
2. Can prevent double speeding attack
3. High scalability: The account-based model has better scalability when processing a large number of transactions. Since there is no need to track the UTXO of each transaction, transaction processing speed is relatively fast, thus increasing the throughput of the entire network.

Disadvantages of the account based model

1.replay attack (nonce counter)

2. In account mode, the balance and status information of each account needs to be stored. As the number of accounts and transaction volume increases, larger storage space is required to save information for all accounts, which may result in increased storage costs.

Account Category

Externally owned account

• balance
• nonce

External accounts are also called EOA accounts or user accounts. They are standard accounts generated and controlled by users, using private and public keys to implement identity authentication and transaction signatures.

• Each external account has a corresponding Ethereum (ETH) balance and can send and receive Ether.
• External accounts can be used to participate in ordinary currency transactions, such as transferring Ether to other accounts.
• The external account signs the transaction via the private key, which is then broadcast to the blockchain network.

Smart contract account

• balance
• nonce (one contract can call another contract, so the number of calls is recorded through nonce)
• code
• storge (the calling process will change)

Contract accounts are special accounts controlled by smart contract code. They do not rely on private and public keys for identity authentication.

• A smart contract is a programmatically defined, self-executing computer code capable of executing transactions or implementing specific functions based on preset rules and conditions.
• Smart contract code is deployed to Ethereum, generating a contract account and allocating an Ether balance to cover the cost of transaction execution (called "gas" fees).
• Contract accounts can handle complex business logic and multi-party interactions, and can implement more complex decentralized applications (DApps).

Smart contracts require stable account participation

State tree in Ethereum

Idea 1: Use hash table query

There is a problem: every time a new transaction appears, it is packaged into a block, thereby changing the Merkle tree, but in fact only part of it changes, and part of the change requires the entire Merkle tree to be rewritten.

(Also due to BTC)

Idea 2: Directly use the Merkle tree to access accounts. Is it feasible to change the merkle tree directly?

• Merkle tree search efficiency is low
• Without sorting, there is a problem: when calculating the order of leaf nodes, the order cannot be reached (the order of BTC is determined by the node with the right to mint)

What if you use sorted merkle tree?

When inserting an account, the reconstruction of the merkle tree is very expensive.

MTP structure

Trie structure (Trie is a tree-like data structure used to store and quickly retrieve key-value pairs, usually used to efficiently store large amounts of associated data.) It can be compared to the state transfer of a signal

Features of trie structure:

1. Shuffle the order, and the trie structure remains unchanged.
2. Has good locality of update operations

Disadvantages of trie: waste of storage, low efficiency of some content (sparse distribution of key values)

Patricia Tree (compressed prefix tree) is a tree that has undergone path compression

The height of the tree is significantly reduced (more obvious if the tree is sparsely distributed)

disintermediarion(disintermediation)

In order to prevent hash collision, the address is set to $2^{160}$ is so big

Merkle Patricia Tree

Replace ordinary pointers with hash pointers

Root Node: The root node of the state tree is a hash value used to represent the current state of the entire state tree. All account status data is uniquely identified by the hash value of this root node.

Branch Node: A branch node is an intermediate node in the state tree, used to connect different leaf nodes or other branch nodes. Each branch node has 17 child nodes, corresponding to hexadecimal numbers 0-15 and a special node (NIL node). Each child node saves a hash value pointing to the next level branch node or leaf node.

Leaf Node: The leaf node contains the specific status information of the account. Each leaf node corresponds to an account address and stores account status data related to the address, such as balance, code, storage, etc.

The structure of this state tree enables Ethereum to efficiently track and maintain the state of accounts, and also helps implement functions such as light client verification and state proof, thereby improving the scalability and security of the entire network.

The actual structure used by Ethereum ismodified MPT

The root node hash value exists in the block header

Each time a block is published, the value of the new node in the state tree will change, and these changes are local.

shared branch

Question: Why retain the historical state and not modify the original data?

Answer: In order to roll back, it is normal to fork in ETH. The data in the orphan block must be rolled back to the previous state. Since there are smart contracts in ETH, in order to support the rollback of smart contracts, the previous state must be maintained.

Block header structure

Published information

The state tree saves (key, value). The above mentioned addresses are all the addresses where keys are saved. So what about Value?

Serialized and stored after RLP (Recursive Length Prefix)

Only do nested array of bytes

Blockchain transaction tree and receipt tree

State tree: a state containing multiple blocks

The role of transaction tree and receipt tree (transactions and transaction results of independent blocks):

1. Provide merkle proof

2. Supports more complex query operations ->eg supports searching for transactions related to a smart contract in the past 10 days

   Solution: Introduce the bloom filter data structure (false positives may occur, but false passives will not occur)

bloom filter data structure

Bloom Filter is a probabilistic data structure used to quickly determine whether an element belongs to a certain set. It retrieves elements efficiently while taking up less memory space. The main application scenario of Bloom filter is to perform fast search and filtering operations in large-scale data sets.

ETH’s specific operations on bloom filter

After each transaction is executed, a receipt will be formed. The receipt contains a bloom filter to record the transaction type, address and other information.

The published block also has a general bloom filter in its block header. This general bloom filter is the bloom filter of all transactions in the block.union

When we need to find all transactions related to smart contracts in the past 10 days, the method is:

​ Step1: Find the bloom filter in the block header of which block has the type of transaction we need. If it does not exist in the block header, then this block is not the block we need.

Step2: If there is a block header, go to this block to find the bloom filter of each receipt, and if there is one, confirm the transaction.

ETH operation process: transaction-driven state machine (tx-driven state machine)

Q: In the current state tree mechanism, the state that contains the information of all accounts can be changed to the state that only contains the accounts with tx involved in the block?

1. The transactions involved in each node may be incomplete, and it is inconvenient to find the status of an account.

The design of the current state tree ensures that the state of each block is complete and consistent. If only the account status of the transactions involved in the block is included, there may be information loss or inconsistency, resulting in network insecurity.

2. It is troublesome to search for a new account. The worst is to find the genesis block to find out that it is a new account.

GHOST protocol

Forks are the norm in Ethereum

The possible harm caused by ETH setting the block generation time to 10 seconds

But for ETH, the block generation time is too short. If you continue to use the longest legal chain, it will be especially unfair to individual miners, because large mines have the ability to create the longest legal chain.

mining pool

The core of the GHOST protocol: giving "consolation prizes" to nodes that mine orphan blocks

Consider forked blocks: In the GHOST protocol, when a node selects the longest chain, it not only considers the blocks on the longest chain, but also considers other forked blocks (that is, blocks that are not on the longest chain). Specifically, nodes choose the heaviest branch chain, not just the longest chain.

Reward uncle blocks: In the GHOST protocol, uncle blocks (also called uncle nodes) refer to those forked blocks (including up to two uncle blocks) that are not selected as the final block. Although uncle blocks are not included in the longest chain, they also contribute positively to blockchain security and fork issues. The GHOST protocol will give certain rewards to miners of uncle blocks to encourage miners to choose a wider range of blocks and increase the overall security of the network.

Uncle Block can continue forever, and the rewards can be extended for up to 7 generations.

state fork

Question: If the tx of uncle block is included, will the tx be executed?

Answer: No execution, the transaction may be repeated, and the legality of the uncle block transaction is not checked.

Question: What should I do if the uncle block is followed by a string? Is it considered an uncle block?

ETH actual

Please refer to the tx on the etherscan website

Ethereum’s mining algorithm

Ethash

Block chain is secured by mining

The controversial aspect of BTC mining is that it requires the use of professional ASIC chips, which runs counter to the concept of decentralization.

An important method of ASIC resistance is memory hard mining puzzle

Example: Litecoin (based on Scrypt)

ETH improvements, 2 data sets

Ethash is a heavyweight PoW algorithm used by many of the most popular cryptocurrencies, including Ethereum, as well as MOAC, Expanse, Pirl, and more. This algorithm is different because it uses DAG files, which are loaded into the GPU memory at the moment the miner starts. An epoch change occurs every 30,000 blocks, increasing the DAG file by 8MB.

ETHASH has improved Dagger-Hashimoto, effectively solving the dilemma of purely memory-dependent algorithms such as the Scrypt algorithm, which is as difficult to encrypt as it is to decrypt. It also breaks through the dilemma of the Dagger algorithm not resisting memory sharing hardware acceleration, changing the generation of full blockchain data. For the generation of a fixed 1GB of data, the client supports pre-generated data to ensure a smooth transition of mining difficulty.

16M cache -> light node for easy verification

1G dataset -> full node

Specific process

The 16M cache generation method is similar to Scrypt

step0 calculates a seed (Seed). The calculation of the seed depends on the current block and all blocks before this block.

step1 Fill in the random number (nonce) at the first position in the array, calculate the subsequent numbers using the hash function, and fill in the elements in the cache in turn (256 bits)

A cache is calculated from the seed, which is a 16MB data set derived only from the seed. Light clients SHOULD store this cache for later verification.

step2 get 256 hashes and put them into a larger array (1G dataset)

step3 Find the nonce, read the corresponding position (and loop) through the nonce, go back and forth 64 times, find 128 numbers, get the hash, and solve the puzzle

The mining process is the hashing process. The input to the hash is to take 128 subparts of a 1GB dataset and put them together to perform the hash. The verification process is to generate the mined data fragments through the Cache cache on the light client and perform hash verification. Therefore, the light client does not need to save the 1GB DAG at all times.

Pseudocode analysis

step 1

#step1 生成16Mcache
def mkcache(cache_size,seed):
    o=[hash(seed)]
    for i in range(cache_size):
        o.append(hash(o[-1]))#hash函数 o[-1]是上一个hash值
    return o

step 2

#step2 通过cache生成dataset中的第i个数据
def calc_dataset_item(cache,i):
    cache_size=cache.size#cache的大小
    mix=hash(cache[i%cache_size]^i)#cache[i%cache_size]^i是一个随机数
    for j in range(256):
        cache_index=get_int_from_item(mix)
        mix=make_item(mix,cache[cache_index%cache_size])
    return hash(mix)

Step 3: Call this function multiple times to get the complete dataset

puzzle algorithm implementation

Why does the miner need to save the entire dataset, but the plot points only need to be saved in the cache?

Ethereum total supply

https://etherscan.io/stat/supply

Ethereum mining difficulty adjustment

There are many discrepancies in the ETH mining difficulty adjustment operation, so the official code is mainly used here.

PART 1: Main formula

PART 2

Parameter description: $x$ is the unit of adjustment,$g_2$is the adjustment factor

$H_s$is the current timestamp, $P(H{)}_{H_S}$is the timestamp of the parent block, and the time difference between the two is the block interval.

PART 3

Parameter Description

Proof of Stake

POW has been criticized for wasting resources

Question: Is the energy consumed by mining necessary?

The essence of mining: everyone competes for computing power. The greater the computing power, the higher the mining probability. Therefore, the essence is: the more money invested, the higher the mining probability.

POS core idea: fight for money directly -> virtual mining

POS vs. POW advantages:

1. energy saving
2. POS is a closed-loop ecosystem, while POW is an open ecosystem, so POS naturally prevents 51% attacks (the attacker must purchase enough cryptocurrency (become a shareholder or obtain more voting rights) to have the ability to launch an attack, but at this time, It benefits the developers and early miners of the currency, just like a hostile takeover)

noting at state: When the following forks occur, POW will choose one chain, and under POS, both chains can bet (without loss)

The POS protocol that ETH plans to adopt is Casper the Friendly Finality Gadget (FFG)

Introduced: validator->invest a certain amount of deposit

Every 100 blocks are mined to generate an epoch, and then voting begins (two-phase commit)

There are two rounds of voting required in the voting (both must reach $\frac{2}{3}$)

Only vote once per epoch (validator vote)

For validators, active participation in voting will be rewarded, while administrative inaction will be punished (deduction of deposit).

For the behavior of betting on both sides, all deposits will be confiscated.

Ethereum Virtual Machine

The most important activity on Ethereum, besides transferring money, is compiling and running smart contracts.

Smart contracts represent an independent steward in the Ethereum world. It carries out Ethereum income and expenditure activities according to its own code instructions, and also has a certain storage space to store some data.

Ethereum virtual machine execution is divided into two major categories, read-only operations and write operations. Operations that only obtain the blockchain status are read-only operations.

The read-only operation does not modify the blockchain state, nor does it trigger any state changes when calling the contract in the chain, so it is relatively fast.

Write operations change the state of the blockchain. For example, an operation that changes the account status.

A write operation requires Ethereum because it changes the storage space of one or several accounts. Storage space is part of the blockchain and is to be stored permanently and synchronously by computers around the world. This change process is expensive: for example, changing a value from to a non-zero value 0requires 20000units of gas; modifying a 非0value requires 5000units of gas . gas; assigning a value from 非0to 0can recycle 15000units of gas; reading a variable value requires 200gas. In contrast, reading variable values ​​from memory only requires 3gas.

The Ethereum virtual machine does not contain the hardware registers that normal CPUs have. Its execution width is a fixed-length value of 256 bits, so it is relatively easy to program. It contains three major storage mechanisms: storage, stack, and memory.

Storage

The values ​​in storage are permanently recorded on the blockchain. Storage is expensive to write and read, so data should not be stored in storage unless absolutely necessary.

The read and write operations in the storage area are based on 256bit, and there is no smaller operation space. Therefore, uint8 and uint256 occupy the same space and consume the same Gas when storing single values. After wrapping unit8 into the struct structure, you can save space and gas expenses through optimization.

Stack stack

The stack has only 1024 layers of depth. When we execute too many recursive calls, the stack will penetrate 1024 layers and the code will fail to execute.

Only the upper 16 layers of the stack can be accessed quickly. The width of the stack is also 256 bits, which is 32 bytes, the length of a word. Any read and write operations are performed in units of 256 bits. Compilers often put temporary variables and variable addresses during code execution on the stack for temporary storage. The variable addresses can be further indexed into memory.

Memory memory

Virtual machine input and output

Gas cost

When sending a transaction, we can specify the upper limit of gas consumption to prevent bugs in the smart contract code from causing infinite loop execution.

Once the gas is exhausted prematurely, the virtual machine throws an exception and ends code execution.

There is a very special situation, which is assert()the function and require()function of the Solidity smart contract code.

During execution, both functions judge the true and false conditions, but the assert function is more emotional and often judges key security conditions. For example, in SafeMath, the assert function is used to judge whether the number of digits has overflowed; when withdrawing from the contract, it is judged Whether the caller is the contract owner, etc.

The require() function is a more common conditional judgment, such as judging whether the balance of the contract caller is sufficient, etc.

Once the assert() function fails to judge, all remaining gas will be deducted as a punitive measure; if the require() fails to judge, it will only stop the current execution, charge the gas fee corresponding to the current step, and then undo the changes.

Virtual machine instruction set

The EVM executes bytecode . Since opcodes are limited to one byte, the EVM instruction set can only accommodate up to 256 instructions.

mem[a...b] represents the bytes a to b (excluding b) in the memory.
storage[p] represents the 32 bytes starting from p
. Remember that the word of the evm virtual machine is 256 bits and 32 bytes.

smart contract

What is a smart contract?

A smart contract is a piece of code that runs on the blockchain. The logic of the code defines the content of the contract. The
account of the smart contract saves the current running status of the contract.

• balance: current balance
• nonce: number of transactions
• code: contract code
• storage: storage, the data structure is an MPT
• Solidity is the most commonly used language for smart contracts, and its syntax is very close to JavaScript.

All functions in Ethereum that need to accept external transfers need to be marked aspayable

Solidity language does not support hash table traversal

bidders.push(bidder)//添加竞拍人
bidders.length//大小

Call smart contract

Create a transaction, the receiving address is the address of the smart contract to be called, and the data field is filled with the encoded values ​​of the function to be called and its parameters.

1. Call directly

3.Agent call

The delegate calls delegatecall()

The usage method is the same as call(), except that .value() cannot be used.
The difference lies in whether to switch context

• call0 switches to the called smart contract context
• delegatecall() only uses the code of the given address, and other properties (storage, balance, etc.) are taken from the current contract. The purpose of delegatecall is to use library code stored in another contract.

fallback() function

function()public [payable]{
......
}

• Anonymous functions have no parameters and no return value.
• It will be called in two situations:
  • Transfer funds directly to a contract address without adding any data
  • The called function does not exist
• If the transfer amount is not 0, you also need to declare payable, otherwise an exception will be thrown.

Creation and operation of smart contracts

• After the smart contract code is written, it must be compiled into bytecode
• Create a contract: The external account initiates a transfer transaction to the address 0x0
  • The amount transferred is 0, but you have to pay for gas
  • The code of the contract is placed in the data field
• Smart contracts run on EVM (Ethereum Virtual Machine).
  Ethereum is a transaction-driven state machine.
  • After a transaction calling a smart contract is published to the blockchain, each miner will execute the
    transaction and deterministically move from the current state to the next state.

Gas fee

ETH error handling

ETH transactions are atomic, that is, a transaction will either not be executed, or all of it will be executed, with no part of it executed.

• There is no custom try-catch structure in smart contracts

• Once an exception is encountered, all execution operations will be rolled back except under special circumstances.

• Statements that can throw errors

  • assert(bool condition): Thrown if the condition is not met - used for internal errors.

  • require(bool condition): Throw if the condition is not met - used for errors caused by input or external components.

    Function bid()pubiie payable{
    //拍卖未结束
    require(now <=auctionEnd);
    }
    

  • revert(): Terminate running and rollback status changes (unconditional)

Nested calls

Block header

Gasoline fee deduction mechanism

1. The three trees of ETH are all locally maintained data structures.
2. The full node receives the call to the smart contract and deducts the balance from the local account.
3. Then the node that has obtained the accounting rights will publish the block and upload the three trees it maintains locally to the blockchain.

ETH mining process

1. Full node package transactions, execute calls to smart contracts, and adjust the content of smart contracts
2. Find the root hash value of 3 trees
3. Try nonce, obtain accounting rights, and publish blocks
4. Other blocks that have not obtained accounting rights must independently verify the legality of the newly released blocks and the tx and smart contracts they contain.

It should be executed first and then mined (the hash of three trees needs to be calculated first and then written into the block)

Without verification, the hash value of the local three trees cannot be updated, the correct three trees information cannot be published, and mining cannot be done.

Receipt

Question: Does smart contract support multi-threading?

Not supported, the state machine must be completely deterministic, and if multiple threads access memory in different sequences, the results may be different.

Out of multithreading, "generate random numbers"

Block information available to smart contracts

block.blockhash(unint blockNumber) returns (bytes)The hash of a given block - only valid for the most recent 256 blocks and does not include
block.coinbase(address)the address of the miner who mined the current block
block.difficulty(uint)Current block difficulty
block.gaslimit(uint): Current block Gas limit
block.number(uint): Current block number
block.timestamp(uint): Since unix epoch Timestamp in seconds starting from the current block

The call information that the smart contract can obtain
msg.data(bytes) : complete calldata
msg.gas(uint): remaining gas
msg.sender(address): message sender (current call)
msg.sig(bytes4): first 4 bytes of calldata (that is, function identifier)
msg.value(uint): number of wei sent with the message
now(uint): current block time Stamp (block.timestamp)
tx.gasprice(uint): the gas price of the previous transaction
tx.origin(address): the initiator of the previous transaction (complete call chain)

Address type

<address>.balance(uint256)
The balance of the address type in Wei.
<address>.transfer(uint256 amount):
Send an amount of Wei to the address type. If it fails, an exception will be thrown. A mining fee of 2300 (the address transferred to the account) gas will be sent, which cannot be adjusted.

<address>.send(uint256 amount)returns(bool):
Send an amount of Wei to the address type, return false if failed, and send a miner fee of 2300 gas, which cannot be adjusted.

<address>.call(...)returns (bool):
Issue the underlying CALL, returns false on failure, sends all available gas, cannot be adjusted.

<address>.callcode(...)returns(bool)Issue
the underlying CALLCODE, return false on failure, send all available gas, and cannot be adjusted.

<address>.delegatecall(...)returns (bool):
Issue the underlying DELEGATECALL, return false on failure, send all available gas, and cannot be adjusted.

All smart contracts can be explicitly converted to address types

Three ways to send ETH

<address>.transfer(uint256 amount)
<address>.send(uint256 amount)returns (bool)
<address>.call.value(uint256 amount)()

Simple case

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract SimpleAuction {
    // 拍卖的受益人
    address payable public beneficiary;
    // 最高出价者
    address public highestBidder;
    // 最高出价
    uint public highestBid;

    // 拍卖结束时间
    uint public auctionEndTime;

    // 竞标是否已结束
    bool public ended;

    // 变更出价时触发的事件
    event HighestBidIncreased(address bidder, uint amount);

    // 拍卖结束时触发的事件
    event AuctionEnded(address winner, uint amount);

    // 构造函数，设置受益人和拍卖结束时间
    constructor(uint _biddingTime, address payable _beneficiary) {
        beneficiary = _beneficiary;
        auctionEndTime = block.timestamp + _biddingTime;
    }

    // 竞标函数
    function bid() public payable {
        require(block.timestamp <= auctionEndTime, "拍卖已结束");
        require(msg.value > highestBid, "已存在更高出价");

        if (highestBid != 0) {
            // 返还之前最高出价者的金额
            address payable previousBidder = payable(highestBidder);
            previousBidder.transfer(highestBid);
        }

        highestBidder = msg.sender;
        highestBid = msg.value;
        emit HighestBidIncreased(msg.sender, msg.value);
    }

    // 结束拍卖函数，只有受益人可以调用
    function auctionEnd() public {
        require(block.timestamp >= auctionEndTime, "拍卖尚未结束");
        require(!ended, "拍卖已结束");

        ended = true;
        emit AuctionEnded(highestBidder, highestBid);

        // 将最高出价转给受益人
        beneficiary.transfer(highestBid);
    }
}

references

ETHASH Mining Algorithm — Ethereum’s Compass 1.0.0 documentation (abyteahead.com)

Virtual machine execution resources — Ethereum’s Compass 1.0.0 documentation (abyteahead.com)