1. What is high concurrency
High concurrency is one of the factors that must be considered in the design of the Internet distributed system architecture. It usually means that the design ensures that the system can process many requests in parallel at the same time.
Some commonly used indicators related to high concurrency include Response Time, Throughput, QPS (Query Per Second), and the number of concurrent users.
Response time : The time it takes for the system to respond to the request. For example, it takes 200ms for the system to process an HTTP request, and this 200ms is the response time of the system.
Throughput : The number of requests processed per unit time.
QPS : The number of response requests per second. In the Internet domain, the distinction between this metric and throughput is not so clear.
Number of concurrent users : The number of users who are using the system functions normally at the same time. For example, in an instant messaging system, the number of simultaneous online users represents the number of concurrent users of the system to a certain extent.
2. How to improve the concurrency capability of the system
There are two main methods to improve the concurrency capability of the Internet distributed architecture design: vertical expansion (Scale Up) and horizontal expansion (Scale Out).
Vertical expansion : Improve the processing capacity of a single machine. There are two ways to scale vertically:
(1) Enhance the hardware performance of a single machine, for example: increase the number of CPU cores such as 32 cores, upgrade a better network card such as 10 Gigabit, upgrade a better hard disk such as SSD, expand the hard disk capacity such as 2T, and expand the system memory such as 128G;
(2) Improve the performance of the stand-alone architecture, for example: use Cache to reduce the number of IOs, use asynchrony to increase the throughput of a single service, and use a lock-free data structure to reduce the response time;
In the early days of the rapid development of the Internet business, if budget is not a problem, it is strongly recommended to use the method of "enhancing the performance of single-machine hardware" to improve the concurrency capability of the system, because at this stage, the company's strategy is often to seize time for business development, and "enhance the performance of single-machine hardware" "It's often the fastest way.
Whether it is to improve the performance of stand-alone hardware or to improve the performance of stand-alone architecture, there is a fatal deficiency: stand-alone performance is always limited. Therefore, the ultimate solution for the design of high concurrency in the distributed architecture of the Internet is horizontal expansion.
Horizontal expansion : As long as the number of servers is increased, the system performance can be linearly expanded. Horizontal expansion requires system architecture design. How to design horizontal scalability at each layer of the architecture, as well as the common horizontal expansion practices at each layer of the Internet company architecture, are the focus of this article.
3. Common Internet Layered Architecture
The common Internet distributed architecture is as above, divided into:
(1) Client layer : The typical caller is a browser browser or a mobile application APP
(2) Reverse proxy layer : system entry, reverse proxy
(3) Site application layer : implement core application logic and return html or json
(4) Service layer : if the service is realized, there is this layer
(5) Data - cache layer : cache accelerates access to storage
(6) Data - database layer : database curing data storage
How to implement the horizontal expansion of the whole system at all levels?
Fourth, the practice of hierarchical horizontal expansion architecture
Horizontal scaling of the reverse proxy layer
The horizontal expansion of the reverse proxy layer is achieved through "DNS polling": dns-server configures multiple resolution IPs for a domain name, and each time a DNS resolution request is made to access dns-server, these IPs will be polled and returned.
When nginx becomes the bottleneck, as long as the number of servers is increased, the deployment of the nginx service is added, and an external network IP is added, the performance of the reverse proxy layer can be expanded, and the theoretically infinitely high concurrency can be achieved.
Horizontal scaling of the site layer
The horizontal expansion of the site layer is achieved through "nginx". By modifying nginx.conf, multiple web backends can be set up.
When the web backend becomes a bottleneck, as long as the number of servers is increased, the deployment of new web services is added, and a new web backend is configured in the nginx configuration, the performance of the site layer can be expanded, achieving theoretically infinitely high concurrency.
Horizontal scaling of the service layer
The horizontal expansion of the service layer is realized through the "service connection pool".
When the site layer calls the downstream service layer RPC-server through RPC-client, the connection pool in the RPC-client will establish multiple connections with the downstream service. When the service becomes a bottleneck, just increase the number of servers and deploy new services. Establishing a new downstream service connection at the RPC-client can expand the performance of the service layer and achieve theoretically infinitely high concurrency. If you need to perform automatic expansion of the service layer elegantly, you may need to support the automatic service discovery function in the configuration center.
Horizontal scaling of the data layer
In the case of a large amount of data, the data layer (cache, database) involves horizontal expansion of data, and the data (cache, database) originally stored on one server is horizontally split to different servers to expand system performance. the goal of.
There are several common horizontal splitting methods in the Internet data layer. Take the database as an example:
Split horizontally by range
Each data service stores a certain range of data. The above figure is an example:
user0 library, storage uid range 1-1kw
user1 library, storage uid range 1kw-2kw
The benefits of this program are:
(1) The rules are simple, the service can route to the corresponding storage service only by judging the uid range;
(2) The data balance is better;
(3) It is relatively easy to expand, and a data service of uid [2kw, 3kw] can be added at any time;
The shortcomings are:
(1) The load of the request is not necessarily balanced. Generally speaking, the newly registered users will be more active than the old users, and the service request pressure of the large range will be greater;
Split by hash level
Each database stores part of the data hashed by a certain key value. The above figure is an example:
user0 library, storing even uid data
user1 library, storing odd uid data
The benefits of this program are:
(1) The rules are simple, the service only needs to hash the uid to route to the corresponding storage service;
(2) The data balance is better;
(3) The request uniformity is better;
The shortcomings are:
(1) It is not easy to expand. To expand a data service, when the hash method is changed, data migration may be required;
It should be noted here that the expansion of system performance through horizontal splitting is fundamentally different from that of master-slave synchronous read-write separation to expand database performance.
Scale database performance by splitting horizontally:
(1) The amount of data stored on each server is 1/n of the total, so the performance of a single machine will also be improved;
(2) There is no intersection of data on n servers, and the union of data on that server is the complete set of data;
(3) The data is horizontally split to n servers. In theory, the read performance is expanded by n times, and the write performance is also expanded by n times (in fact, it is far more than n times, because the data volume of a single machine has become the original 1/n);
Extend database performance through master-slave synchronous read-write separation:
(1) The amount of data stored on each server is the same as the total;
(2) The data on the n servers are all the same, and they are all complete sets;
(3) Theoretically, the read performance is expanded by n times, the write is still a single point, and the write performance remains unchanged;
The horizontal splitting of the cache layer is similar to the horizontal splitting of the database layer, and most of them are range splitting and hash splitting, so they will not be expanded.
V. Summary
High concurrency is one of the factors that must be considered in the design of the Internet distributed system architecture. It usually means that the design ensures that the system can process many requests in parallel at the same time.
There are two main methods to improve the concurrency capability of the system: vertical expansion (Scale Up) and horizontal expansion (Scale Out). The former vertical expansion can improve concurrency by improving the performance of single-machine hardware or improving the performance of single-machine architecture, but there is always a limit to single-machine performance. The ultimate solution for high concurrency in Internet distributed architecture design is the latter: horizontal expansion.
In the layered architecture of the Internet, the practice of horizontal expansion at each layer is different:
(1) The reverse proxy layer can be extended horizontally by means of "DNS polling";
(2) The site layer can be extended horizontally through nginx;
(3) The service layer can be extended horizontally through the service connection pool;
(4) The database can be horizontally expanded according to the data range or data hash;
After the horizontal expansion of each layer is implemented, the performance of the system can be improved by increasing the number of servers, so that the theoretical performance is unlimited.
At the end, I hope that the idea of Is it no longer mysterious?
==【End】==