overview
From 0, study Golang and Golang's microservice ecosystem. Golang's microservices first start from Rpc, and then upgrade to Grpc, and introduce in detail what technical problems these technical points are solving.
Rpc
- Rpc (Remote Procedure Call) remote procedure call, the simple understanding is that one node requests the service provided by another node.
- Corresponding to Rpc is a local procedure call, and the most common local procedure call is a function call.
- Turning a local procedure call into a remote procedure call faces various problems.
Problems faced by the remote call process?
1. Call ID mapping.
How do we tell the remote machine the function ID we want to call? In the local call, the function body is directly specified through the pointer. When we call the function, the compiler will automatically call its corresponding function pointer for us.
But in remote calls, function pointers are not acceptable, because the address spaces of the two processes are completely different. Therefore, in RPC, all functions must have their own ID. This ID is uniquely determined across all processes. The client must attach this ID when making a remote call. Then, the client and the server need to maintain a corresponding table of {function <–> Call ID} respectively. The two tables do not necessarily need to be exactly the same, but the Call ID corresponding to the same function must be the same.
When the client needs to make a remote call, it checks the table, finds the corresponding Call ID, and then passes it to the server. The server also checks the table to determine the function that the client needs to call, and then executes the corresponding function. The code of the function.
2. Serialization and deserialization
How does the client pass the parameter value to the remote function? In the local call, we only need to push the parameter to the stack, and then let the function read it from the stack by itself. However, in remote procedure calls, the client and the server are different processes, and parameters cannot be passed through memory. Even sometimes the client and the server do not use the same programming language. At this time, the client and the server need to convert the parameters into a byte stream first, and then transfer the byte stream into a byte stream that they can understand. format, this process is called serialization and deserialization.
3. Network transmission
Remote calls are often used on the network, and the client and server are connected through the network. All data needs to be transmitted over the network, so a network transport layer is required. The network transport layer needs to stream the Call ID and serialized parameter bytes to the server, and then return the serialized call result to the client. As long as it can complete both, it can be used as the transport layer.
Therefore, the protocol it uses is actually unlimited, as long as it can complete the transmission. Although most Rpc frameworks use the Tcp protocol, Udp can also be used, and gRPC simply uses Http2.
What problems does the Rpc framework need to solve
Problems to be solved on the client side:
1. Map this call to a Call ID. Here we assume that the simplest string is used as the Call ID.
2. Serialize Call ID a and b, and directly pack their values in binary form.
3. Get from 2. The data packet is sent to ServerAddr, which requires the network transport layer
4. Wait for the server to return the result
5. If the server call is successful, it will be deserialized by the result and dedicated to total
Problems solved on the server side:
1. Maintain a mapping call_id_map from Call ID to function pointer locally, which can be completed with dict
2. Waiting for requests, including multi-threaded concurrent processing capabilities
3. After getting a request, deserialize its data packet to get Call ID
4 .Get the corresponding function pointer by searching in call_id_map
5. After deserializing aherb, call the add function locally to get the result
6. Serialize the result and return it to the Client through the network
In the entire process above, it is estimated that some students have seen the familiar computer network process and the definition of web server, so to implement an Rpc framework, in fact, it only needs to be implemented according to the above process and it is basically completed.
in:
- Call ID mapping can directly use function strings or integer IDs, and the mapping table is generally a hash table.
- Serialization and deserialization can be written by yourself, or you can use Protobuf or FlatBuffers.
- The network transmission library can write the socket by itself, or use asio, ZeroMQ, Netty and the like.