Good old guys, this is the Java Research Institute.
Today we will discuss the 4
implementations of distributed locks : 1. Implement distributed locks through MySQL 2. Implement distributed locks
through redis 3. Implement distributed locks
through zookeeper 4. Implement distributed locks
through etcd
1. What is a distributed lock?
How to ensure that shared resources can only be accessed by one thread at a time?
You may think this is very simple, in a jvm, it is very easy to achieve through synchronized or ReentrantLock.
Indeed, there is really no problem in a single JVM.
However, usually our system will be deployed in a cluster. At this time, each node in the cluster is a jvm environment, so the problem of shared resource access cannot be solved through synchronized or ReentrantLock.
Distributed locks are needed at this time: Distributed locks solve the problem of sequential access to shared resources in a distributed environment. At the same time, only one thread of all nodes in the cluster can access shared resources.
2. The function of distributed locks
Distributed lock users are located in different machines. After the lock is successfully acquired, the shared resource can be operated on
. Only one user of all machines at the same time can obtain the distributed lock. The
lock has a reentrant function: that is, one user can
The process of acquiring a distributed lock multiple times allows specifying the timeout function: try to acquire the lock within the specified time, after the timeout period, if the lock has not been acquired, the acquisition fails to
prevent deadlock: such as: A machine acquires After the lock is locked, machine A hangs up before the lock is released, and the lock is not released. As a result, the lock has been occupied by machine A. In this case, the distributed lock must be automatically resolved; the solution: when the lock is held You can add a holding timeout period. After this time, the lock will be automatically released. At this time, other machines will have the opportunity to acquire the lock.
Let's look at the 4 implementations of distributed locks.
3. Method 1: Database method
3.1 Principle
The lock acquisition process
If there are n systems in a cluster environment, each system has a jvm, and each jvm has m threads to acquire distributed locks, then there may be n*m threads to acquire distribution at the same time At this time, the pressure of distributed locks is relatively large. It is meaningless for multiple threads in each jvm to acquire locks at the same time. You can add a local lock to each jvm to acquire distributed locks. Before you need to acquire the JVM local lock, you can try to acquire the distributed lock after the local lock is successfully acquired. At this time, at most n threads in n systems try to acquire the distributed lock. The steps to acquire the lock are mainly two steps:
1、先尝试获取jvm本地锁
2、jvm本地锁获取成功之后尝试获取分布式锁overtime time
When acquiring the lock, you can pass the maximum waiting time for acquiring the lock. Try to acquire the lock multiple times within the specified time. After the acquisition fails, sleep for a while, and then continue to try to acquire until the time runs out.
Lock validity period
You need to specify the validity period when acquiring the lock. The validity period is how long the user wants to use the lock after acquiring the lock, and why is the validity period required?
If there is no expiration date, when the user acquires successfully, the system suddenly shuts down, then the lock cannot be released, and other threads can no longer acquire the lock.
Therefore, a validity period is required. After the validity period, the lock will become invalid and other threads can try to acquire the lock.
Lock up
What is lock renewal?
For example: when the user acquires the lock, the specified validity period is 5 minutes, but after 5 minutes, the user has not finished his work and wants to continue using it for a while, then the life extension function can be used to delay the validity period of the lock.
You can start a sub-thread to automatically complete the operation of renewing the life. For example, the original validity period is 5 minutes, and when it is used for 4 minutes, the renewal life is 2 minutes, then the validity period is 7 minutes. This is relatively simple and you can play as you like.
3.2, prepare sql
create table t_lock(
lock_key varchar(32) PRIMARY KEY NOT NULL COMMENT '锁唯一标志',
request_id varchar(64) NOT NULL DEFAULT '' COMMENT '用来标识请求对象的',
lock_count INT NOT NULL DEFAULT 0 COMMENT '当前上锁次数',
timeout BIGINT NOT NULL DEFAULT 0 COMMENT '锁超时时间',
version INT NOT NULL DEFAULT 0 COMMENT '版本号，每次更新+1'
)COMMENT '锁信息表';Note: There is a version number field in the table. The version number is mainly used to update the data in an optimistic locking manner to ensure the correctness of the updated data under concurrent conditions.
3.3, lock tool code
The code is relatively simple. You mainly look at the lock method for acquiring the lock and the unlock method for releasing the lock. The comments are more detailed, and you can understand it after reading it.
The key point in the code is when updating data, by comparing the version number, using the cas method to ensure the correctness of the updated data under concurrent conditions.
This code implements the operation of acquiring and releasing the lock. The operation of renewing the life is not implemented. You can try to implement it.
package lock;
import lombok.Builder;
import lombok.Getter;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import java.sql.*;
import java.util.Map;
import java.util.Objects;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
@Slf4j
public class DbLockUtil {
//将requestid保存在该变量中
static ThreadLocal<String> requestIdTL = new ThreadLocal<>();
//jvm锁：当多个线程并发获取分布式锁时，需要先获取jvm锁，jvm锁获取成功，则尝试获取分布式锁
static Map<String, ReentrantLock> jvmLockMap = new ConcurrentHashMap<>();
/**
* 获取当前线程requestid
*
* @return
*/
public static String getRequestId() {
String requestId = requestIdTL.get();
if (requestId == null || "".equals(requestId)) {
requestId = UUID.randomUUID().toString();
requestIdTL.set(requestId);
}
log.info("requestId:{}", requestId);
return requestId;
}
/**
* 获取锁
*
* @param lockKey 锁key
* @param lockTimeOut(毫秒) 持有锁的有效时间，防止死锁
* @param getTimeOut(毫秒) 获取锁的超时时间，这个时间内获取不到将重试
* @return
*/
public static boolean lock(String lockKey, long lockTimeOut, int getTimeOut) throws Exception {
log.info("start");
boolean lockResult = false;
/**
* 单个jvm中可能有多个线程并发获取一个锁
* 此时我们只允许一个线程去获取分布式锁
* 所以如果同一个jvm中有多个线程尝试获取分布式锁，需要先获取jvm中的锁
*/
ReentrantLock jvmLock = new ReentrantLock();
ReentrantLock oldJvmLock = jvmLockMap.putIfAbsent(lockKey, jvmLock);
oldJvmLock = oldJvmLock != null ? oldJvmLock : jvmLock;
boolean jvmLockSuccess = oldJvmLock.tryLock(getTimeOut, TimeUnit.MILLISECONDS);
//jvm锁获取失败，则直接失败
if (!jvmLockSuccess) {
return lockResult;
} else {
//jvm锁获取成功，则继续尝试获取分布式锁
try {
String request_id = getRequestId();
long startTime = System.currentTimeMillis();
//循环尝试获取锁
while (true) {
//通过lockKey获取db中的记录
LockModel lockModel = DbLockUtil.get(lockKey);
if (Objects.isNull(lockModel)) {
//记录不存在，则先插入一条
DbLockUtil.insert(LockModel.builder().lock_key(lockKey).request_id("").lock_count(0).timeout(0L).version(0).build());
} else {
//获取请求id，稍后请求id会放入ThreadLocal中
String requestId = lockModel.getRequest_id();
//如果requestId为空字符，表示锁未被占用
if ("".equals(requestId)) {
lockModel.setRequest_id(request_id);
lockModel.setLock_count(1);
lockModel.setTimeout(System.currentTimeMillis() + lockTimeOut);
//并发情况下，采用cas方式更新记录
if (DbLockUtil.update(lockModel) == 1) {
lockResult = true;
break;
}
} else if (request_id.equals(requestId)) {
//如果requestId和表中request_id一样表示锁被当前线程持有者，此时需要加重入锁
lockModel.setTimeout(System.currentTimeMillis() + lockTimeOut);
lockModel.setLock_count(lockModel.getLock_count() + 1);
if (DbLockUtil.update(lockModel) == 1) {
lockResult = true;
break;
}
} else {
//锁不是自己的，并且已经超时了，则重置锁，继续重试
if (lockModel.getTimeout() < System.currentTimeMillis()) {
DbLockUtil.resetLock(lockModel);
} else {
//如果未超时，休眠100毫秒，继续重试
if (startTime + getTimeOut > System.currentTimeMillis()) {
TimeUnit.MILLISECONDS.sleep(100);
} else {
break;
}
}
}
}
}
} finally {
//释放jvm锁，将其从map中异常
jvmLock.unlock();
jvmLockMap.remove(lockKey);
}
}
log.info("end");
return lockResult;
}
/**
* 释放锁
*
* @param lock_key
* @throws Exception
*/
private static void unlock(String lock_key) throws Exception {
//获取当前线程requestId
String requestId = getRequestId();
LockModel lockModel = DbLockUtil.get(lock_key);
//当前线程requestId和库中request_id一致 && lock_count>0，表示可以释放锁
if (Objects.nonNull(lockModel) && requestId.equals(lockModel.getRequest_id()) && lockModel.getLock_count() > 0) {
if (lockModel.getLock_count() == 1) {
//重置锁
resetLock(lockModel);
} else {
lockModel.setLock_count(lockModel.getLock_count() - 1);
DbLockUtil.update(lockModel);
}
}
}
/**
* 重置锁
*
* @param lockModel
* @return
* @throws Exception
*/
private static int resetLock(LockModel lockModel) throws Exception {
lockModel.setRequest_id("");
lockModel.setLock_count(0);
lockModel.setTimeout(0L);
return DbLockUtil.update(lockModel);
}
/**
* 更新lockModel信息，内部采用乐观锁来更新
*
* @param lockModel
* @return
* @throws Exception
*/
private static int update(LockModel lockModel) throws Exception {
return exec(conn -> {
String sql = "UPDATE t_lock SET request_id = ?,lock_count = ?,timeout = ?,version = version + 1 WHERE lock_key = ? AND version = ?";
PreparedStatement ps = conn.prepareStatement(sql);
int colIndex = 1;
ps.setString(colIndex++, lockModel.getRequest_id());
ps.setInt(colIndex++, lockModel.getLock_count());
ps.setLong(colIndex++, lockModel.getTimeout());
ps.setString(colIndex++, lockModel.getLock_key());
ps.setInt(colIndex++, lockModel.getVersion());
return ps.executeUpdate();
});
}
private static LockModel get(String lock_key) throws Exception {
return exec(conn -> {
String sql = "select * from t_lock t WHERE t.lock_key=?";
PreparedStatement ps = conn.prepareStatement(sql);
int colIndex = 1;
ps.setString(colIndex++, lock_key);
ResultSet rs = ps.executeQuery();
if (rs.next()) {
return LockModel.builder().
lock_key(lock_key).
request_id(rs.getString("request_id")).
lock_count(rs.getInt("lock_count")).
timeout(rs.getLong("timeout")).
version(rs.getInt("version")).build();
}
return null;
});
}
private static int insert(LockModel lockModel) throws Exception {
return exec(conn -> {
String sql = "insert into t_lock (lock_key, request_id, lock_count, timeout, version) VALUES (?,?,?,?,?)";
PreparedStatement ps = conn.prepareStatement(sql);
int colIndex = 1;
ps.setString(colIndex++, lockModel.getLock_key());
ps.setString(colIndex++, lockModel.getRequest_id());
ps.setInt(colIndex++, lockModel.getLock_count());
ps.setLong(colIndex++, lockModel.getTimeout());
ps.setInt(colIndex++, lockModel.getVersion());
return ps.executeUpdate();
});
}
private static <T> T exec(SqlExec<T> sqlExec) throws Exception {
Connection conn = getConn();
try {
return sqlExec.exec(conn);
} finally {
closeConn(conn);
}
}
@FunctionalInterface
public interface SqlExec<T> {
T exec(Connection conn) throws Exception;
}
@Getter
@Setter
@Builder
public static class LockModel {
private String lock_key;
private String request_id;
private Integer lock_count;
private Long timeout;
private Integer version;
}
private static final String url = "jdbc:mysql://localhost:3306/dlock?useSSL=false"; //数据库地址
private static final String username = ""; //数据库用户名
private static final String password = ""; //数据库密码
private static final String driver = "com.mysql.jdbc.Driver"; //mysql驱动
/**
* 连接数据库
*
* @return
*/
private static Connection getConn() {
Connection conn = null;
try {
Class.forName(driver); //加载数据库驱动
try {
conn = DriverManager.getConnection(url, username, password); //连接数据库
} catch (SQLException e) {
e.printStackTrace();
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return conn;
}
/**
* 关闭数据库链接
*
* @return
*/
private static void closeConn(Connection conn) {
if (conn != null) {
try {
conn.close(); //关闭数据库链接
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}4. Method 2: Redis method
4.1. Several commands used
-
The
command format of setnx: SETNX key value; it is the abbreviation of "SET if Not eXists" (SET if it does not exist). Only when the key does not exist, the value of the key is set to value. If the key already exists, the SETNX command does nothing. The command returns 1 when the setting is successful, and 0 when the setting fails. -
getset
command format: GETSET key value, set the value of the key to value, and return the old value of the key before it was set. Return value: If the key has no old value, that is, the key does not exist before it is set, then the command returns nil. When the key exists but is not a string type, the command returns an error. - Expire
command format: EXPIRE key seconds, use: set the survival time for a given key, when the key expires (the survival time is 0), it will be automatically deleted. Return value: Return 1 if set successfully. When the key does not exist or the time to live cannot be set for the key (for example, you try to update the time to live of the key in a version of Redis lower than 2.1.3), return 0.
- The del
command format: DEL key [key …], use: delete one or more given keys, the keys that do not exist will be ignored. Return value: the number of deleted keys.
4.2. Principle
For process analysis, first look at the process on the left side of the figure:
1. A tries to obtain the lock key, and sets the lock key through the setnx (lockkey, currenttime+timeout) command, and sets the value to the current time + lock timeout time;
2. If the return value is 1, it means that the redis server is still Without lockkey, that is, no other user owns the lock, A can obtain the lock successfully;
3. Before performing related business execution, execute expire(lockkey) and set the validity period for the lockkey to prevent deadlock; because if the validity period is not set, , Lockkey will always exist in redis. When other users try to acquire the lock, the lock will not be successfully acquired when setnx(lockkey, currenttime+timeout) is
executed ; 4. Perform related business
5. Release the lock, A completes the related business After that, you must release the lock that you own, that is, delete the content of the lock in redis, del (lockkey), and then the user can reset the new value of the lock
Look at the process on the right
6. When A cannot successfully set the lockkey through the setnx(lockkey, currenttime+timeout) command, it cannot be directly concluded that the lock acquisition failed; because we set the lock timeout timeout when setting the lock, when the current time is greater than redis When the storage key is the value of lockkey, it can be considered that the previous owner's right to use the lock has expired, and A can forcibly own the lock; the specific determination process is as follows;
7. A obtains redis through get(lockkey) The storage key value in is the value value of lockkey, that is, the relative time to acquire the lock lockvalueA
8, lockvalueA!=null && currenttime>lockvalue, A compares the current time with the lock setting time, if the current time is greater than the lock setting Time is critical, that is, you can further determine whether the lock can be acquired, otherwise it means that the lock is still occupied, and A cannot acquire the lock yet, and the lock acquisition fails;
9. After the return result in step 4 is true, set a new one through getSet Time out, and return the old value lockvalueB for judgment, because in a distributed environment, when entering here, another process may acquire the lock and modify the value. Only the old value and the returned value are consistent to indicate that the middle is not Other processes acquire this lock
10, lockvalueB == null || lockvalueA==lockvalueB, judgment: if lockvalueB is null, the lock has been released, and the process can acquire the lock at this time; the old value is consistent with the returned lockvalueB It means that the lock is not acquired by other processes in the middle, and the lock can be acquired; otherwise, the lock cannot be acquired, and the lock acquisition fails.
4.3, code
Leave it to everyone, follow the above process to achieve the next.
5. Method 3: zookeeper
5.1. Principle
What is zookeeper? It is an open source middleware that can be used as a high-availability configuration center. Simply understand: it can be used to save some user data.
zookeeper has three important characteristics, to achieve these two features are distributed lock off
button.
The first feature: the nodes are naturally orderly
The data stored in zookeeper is a tree structure, and many nodes can be created under the tree, and user data can be stored in the nodes.
When creating a child node under each node, as long as the selected creation type is an ordered type, then this node will automatically add a monotonically increasing sequence number after the node name specified by the client. The point is that when the child nodes are created concurrently , Can also ensure the orderliness of multiple child nodes.
For example, concurrently create 4 ordered child nodes under /lock/lock1, as follows: the 
client can determine whether the sequence number of the created node is the smallest, and if the number is the smallest among the child nodes, the lock is successfully acquired.
The second feature: temporary nodes
To operate zookeeper, the client needs to establish a connection with zookeeper. If the type of node that the client requests to create on zookeeper is a temporary node, then when the connection between the client and zookeeper is disconnected, the temporary node created will automatically be zookeeper delete.
This can prevent deadlock from multiple functions. For example, if the client hangs up after acquiring the lock, the node will be automatically deleted, and then other lock acquirers have the opportunity to acquire the lock.
The third feature: Listener
The client can add a listener to a node. When the node information changes, zookeeper will notify the client. For example, the node data is modified, the node is deleted, etc., and the client will be notified;
this feature is particularly awesome: this special Cool, the following nodes only need to monitor the one in front of him. When the previous node is deleted, zookeeper will notify the listener. The listener can judge whether the node number he created is the smallest. If it is the smallest, get it The lock is successful. Is this better than the database and redis methods above. The db and redis methods need to spin (the acquisition fails, sleep for a while, continue to cycle attempts), and zookeeper does not need to spin, when the lock is released, Zookeeper will notify the waiter.
5.2, code
Focus on understanding the principle. You can find the code online. There are more, so I won't post it here.
6. Method 4: etcd
Etcd and zookeeper have similar functions and can also be used as a high-availability configuration center. However, etcd is based on the Go language and can also be used to implement distributed locks. The implementation principle is similar to zookeeper, so I won't go into details here.
7. Summary
This article mainly introduces 4 ways to implement distributed locks, everyone should focus on understanding the principle of each way.
The principles of db and redis are similar. When the internal acquisition fails, it needs to use the spin method to retry to acquire the lock, while zookeeper adopts the monitoring method.
The two methods of redis and zookeeper are used more, the performance of redis is better, and the redis method can be used with more concurrency.
There is another point in the design: when acquiring the lock, it is divided into two steps, first acquiring the lock in the jvm, and then trying to acquire the distributed lock.
8. More interview questions
- What are the recommended videos on station B?
- Classic interview question: Why must the hashCode method be rewritten when rewriting the equals method?
- Classic interview question: Why is the default capacity of HashMap 16?
- Classic interview questions: What is the difference between Arraylist and Linkedlist? ? ?
- Classic interview questions: What is the difference between NoClassDefFoundError and ClassNotFoundException?
- Classic interview questions: What is the difference between Throwable, Exception, Error, RuntimeException?
- Classic interview question: How does the code execute when there is a return in try and finally? ? ? ?
- Faced with hundreds of millions of data, whether MySQL is good or not, let's take a look! !
- Classic interview question: ThreadLocal serial gun! !
- Classic interview questions: What is the difference between strong quotes, soft quotes, weak quotes, and false quotes?
- Interviewer: How many states does the thread have? How do they convert to each other?
·END· 
Scan the QR code｜ Follow us