What exactly is zookeeper?
Zookeeper is actually a framework developed by yahoo for distributed consistency processing. Originally it was a by-product of the development of hadoop. Due to the difficulty of consistency processing in distributed systems, other distributed systems do not need to reinvent the wheel. Therefore, zookeeper is widely used in subsequent distributed systems, so that zookeeper has become the basic component of various distributed systems. The importance of status can be imagined. The famous hadoop, kafka, and dubbo are all built on zookeeper.
To understand what zookeeper does, you must first understand what consistency is.
The so-called consistency actually revolves around "seeing". Who can see? Can you see? When did you see it? For example: a Taobao back-end seller puts a big promotion product on the backend and submits it to the main database through server A. Assuming that a user immediately goes to the application server B to query the product from the database, a product will appear. The phenomenon is that the seller has successfully updated, but the buyer cannot see it; after a period of time, the data of the master database is synchronized to the slave database, and the buyer can check it.
Assuming that the buyer can see the seller's update immediately after the seller's update is successful , it is called strong consistency ;
if the buyer cannot see the seller's update after the seller's update is successful, it is called weak consistency ;
After the seller's update is successful, the buyer can finally see the seller's update after a period of time, which is called eventual consistency .
Distributed Transaction Processing Mode
General distributed transaction processing modes include: 2-phase commit, 3-phase commit, TCC (Try-Confirm-Cancel), reliable messages (message queues, database tables), SAGAS long transactions, and compensatory transactions. Which distributed transaction processing mode to use depends on the choice of the appropriate mechanism according to your own business scenario.
Both duboo and spring cloud can be regarded as SOA frameworks, and distributed transaction control support depends on other components, such as through JTA (2-phase, 3-phase), ActiveMQ message middleware, ByteTCC (TCC), etc.
Zookeeper and redis can support distributed locks. Distributed locks are the core of distributed transactions, but distributed locks are not equivalent to distributed transactions.
Redis's support for distributed locks is mainly through setnx. When using redis distributed locks, you must pay attention to the situation where the lock resource is not released normally due to the abnormality of the locked client (for example, the caller is down). When calling setnx, you need to add The judgment on the lock timeout period.
Zookeeper's support for distributed locks can directly use the zookeeper curator-recipes client.
Specific distributed transaction processing
The current solution is the 2pc two-phase commit based on XA, in fact, the entire transaction process is jointly participated by the transaction manager and the resource manager. In this way, the subject may understand that this is a solution at the resource level. To put it bluntly, your database or MQ needs to support the two-phase commit protocol, because resources will be locked throughout the transaction process, and network operations are involved. Then this thing is not very reliable, and it will affect performance, scalability and availability are not friendly, and it is uncertain for the distribution of the service level. Therefore, a tcc was created later, which is similar to 2pc, except that it is located at the business level. The basic idea is to split long distributed transactions into local short transactions. This needs to be designed according to the characteristics of the business. For example, if you buy something for 10 yuan, for the payment of this service, there is an action to freeze the user's 10 yuan when deducting the fee. This is try. The 10 yuan will be deducted when the tcc framework initiates the confirm operation later. If the tcc framework initiates the cancel operation, the 10 yuan will be unfrozen. It should be noted that since confirm and cancel may fail, it can be combined with the global transaction ID to design an idempotent interface.
The above two can provide strong consistency to some extent, but there are many scenarios that do not require such strong consistency. According to the theory of CAP (Consistency, Availability, Reliability), you can't have both, and reliability is a must. Therefore, it is necessary to balance C and A. In fact, A is also necessary in the Internet field. So have to make a fuss on C. So there is weak consistency or eventual consistency, it does not require you to see the effect immediately after doing an operation, as long as you see the correct result in an acceptable time. The architect of epay has introduced this content. At present, the industry uses this kind of more, and the article Sina Visitor System explains it in more detail. The idea of solving distributed transactions is to avoid distributed transactions. Specifically, it is to use local transactions + asynchronous messages + retry + idempotency to ensure the final consistency of the entire system data.