Summary of several methods of implementing distributed locking by Redis
- 2020-05-24 06:26:21
- OfStack
Summary of several methods of implementing distributed locking in Redis
Distributed locking is a way to control the synchronous access of Shared resources between distributed systems. In distributed systems, it is often necessary to coordinate their actions. If one or a group of resources is Shared between different systems or between different hosts of the same system, mutual exclusion is often required to access these resources to prevent interference with each other to ensure 1 uniqueness. In this case, a distributed lock is required.
Let's assume the simplest scenario of seckilling: there is a table in the database, column is the inventory of goods ID, and ID is the corresponding inventory of goods, and the inventory of goods ID is -1 if seckilling is successful. Now let's say I have 1,000 threads to kill two items, 500 threads to kill the first item, and 500 threads to kill the second item. Let's explain 1 distributed lock in terms of this simple business scenario.
Usually, the business system with the second kill scenario is relatively complex, carrying a large amount of business and high concurrency. Such systems often use a distributed architecture to balance the load. So these 1000 concurrences will come from different places, and the inventory of goods is the Shared resource, and it's also the resource that these 1000 concurrences are competing for, so we need to manage the concurrency exclusion. This is the application of distributed locking.
1. Several schemes to realize distributed locking
1.Redis implementation (recommended)
2. Zookeeper implementation
3. Database implementation
Redis Implementing distributed locking
*
* Synchronous processing is often used in multiple servers such as clusters 1 Under the business, this is a common transaction that is unable to meet the business requirements and requires a distributed lock
*
* Common use of distributed locks 3 Kind of implementation:
* 0. Database optimistic lock implementation
* 1.Redis implementation --- use redis the setnx() , get() , getset() Method, used for distributed locking, to solve the deadlock problem
* 2.Zookeeper implementation
* Reference: http://surlymo.iteye.com/blog/2082684
* https://www.ofstack.com/article/103617.htm
* http://www.hollischuang.com/archives/1716?utm_source=tuicool&utm_medium=referral
* 1 , implementation principle:
Based on the zookeeper The main logic of the distributed lock implemented by the instantaneous ordered node is as follows (the figure is from IBM Website). The general idea is that when each client locks a function zookeeper On the directory of the specified node corresponding to this function 1 A wei 1 Is the instantaneous ordered node. The way to determine whether to acquire a lock is simple, only the smallest of the ordered nodes 1 A. When the lock is released, simply delete the transient node. At the same time, it can avoid service downtime caused by the lock can not be released, resulting in deadlock problems.
2 Advantages,
High lock security, zk Be persistent
3 And disadvantages
Performance overhead is high. Because it needs to dynamically create and destroy instantaneous nodes to achieve the lock function.
4 , implementation,
Can be used directly zookeeper The first 3 Party libraries curator You can easily implement distributed locking
*
* Redis Principle of distributed locking:
* 1. through setnx(lock_timeout) Returns if the lock is set 1 . No value has been set to return successfully 0
* 2. Deadlock problem: practice to determine if it is expired, and if it is, get the expiration time get(lockKey) And then getset(lock_timeout) Judge whether or not get The same,
* The same proves that the lock has been locked successfully, because it can cause multiple threads to execute simultaneously getset(lock_timeout) Method, which might result in multiple threads all being required getset Then, for the thread that judged the lock to be successful,
* add expire(lockKey, LOCK_TIMEOUT, TimeUnit.MILLISECONDS) Expiration time, preventing multiple threads from stacking time at the same time, resulting in double lock aging time
* 3. Time for cluster servers is not 1 To the problem, can be called redis the time() Get the current time
2.Redis is implemented in distributed lock code
1. Define the lock interface
package com.jay.service.redis;
/**
* Redis Distributed lock interface
* Created by hetiewei on 2017/4/7.
*/
public interface RedisDistributionLock {
/**
* Locks successfully, returns the lock time
* @param lockKey
* @param threadName
* @return
*/
public long lock(String lockKey, String threadName);
/**
* Unlock, You need to update the lock time to see if you have permissions
* @param lockKey
* @param lockValue
* @param threadName
*/
public void unlock(String lockKey, long lockValue, String threadName);
/**
* Multi-server clusters, using the following method, instead System.currentTimeMillis() To obtain redis Time to avoid multiple service times not 1 Problem!!
* @return
*/
public long currtTimeForRedis();
}
2. Define lock implementations
package com.jay.service.redis.impl;
import com.jay.service.redis.RedisDistributionLock;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.dao.DataAccessException;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.RedisSerializer;
import java.util.concurrent.TimeUnit;
/**
* Created by hetiewei on 2017/4/7.
*/
public class RedisLockImpl implements RedisDistributionLock {
// Lock timeout time in milliseconds, That is, the operation is completed within the lock time, and there will be concurrency if the operation is not completed
private static final long LOCK_TIMEOUT = 5*1000;
private static final Logger LOG = LoggerFactory.getLogger(RedisLockImpl.class);
private StringRedisTemplate redisTemplate;
public RedisLockImpl(StringRedisTemplate redisTemplate) {
this.redisTemplate = redisTemplate;
}
/**
* lock
* Get the lock, get the lock, can't get the lock 1 Wait until you get the lock
* @param lockKey
* @param threadName
* @return
*/
@Override
public synchronized long lock(String lockKey, String threadName) {
LOG.info(threadName+" Start performing locking ");
while (true){ // Loop lock
// Lock time
Long lock_timeout = currtTimeForRedis()+ LOCK_TIMEOUT +1;
if (redisTemplate.execute(new RedisCallback<Boolean>() {
@Override
public Boolean doInRedis(RedisConnection redisConnection) throws DataAccessException {
// Define the serialization mode
RedisSerializer<String> serializer = redisTemplate.getStringSerializer();
byte[] value = serializer.serialize(lock_timeout.toString());
boolean flag = redisConnection.setNX(lockKey.getBytes(), value);
return flag;
}
})){
// If the lock is added successfully
LOG.info(threadName +" Locking success ++++ 111111");
// Set timeout time to free memory
redisTemplate.expire(lockKey, LOCK_TIMEOUT, TimeUnit.MILLISECONDS);
return lock_timeout;
}else {
// To obtain redis Inside time
String result = redisTemplate.opsForValue().get(lockKey);
Long currt_lock_timeout_str = result==null?null:Long.parseLong(result);
// The lock has failed
if (currt_lock_timeout_str != null && currt_lock_timeout_str < System.currentTimeMillis()){
// If it is null or not, it means that it has been invalidated. If it is set by other threads, it will be changed 2 The conditional judgment cannot be executed
// To get on 1 Set the current lock expiration time
Long old_lock_timeout_Str = Long.valueOf(redisTemplate.opsForValue().getAndSet(lockKey, lock_timeout.toString()));
if (old_lock_timeout_Str != null && old_lock_timeout_Str.equals(currt_lock_timeout_str)){
// When you run multiple threads, multiple threads sign up and they all get here, but only 1 The thread has the same set value as the current value before it has the right to acquire the lock
LOG.info(threadName + " Locking success ++++ 22222");
// Set the timeout to free memory
redisTemplate.expire(lockKey, LOCK_TIMEOUT, TimeUnit.MILLISECONDS);
// Returns the lock time
return lock_timeout;
}
}
}
try {
LOG.info(threadName +" Waiting to be locked, sleep 100 ms ");
// TimeUnit.MILLISECONDS.sleep(100);
TimeUnit.MILLISECONDS.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
/**
* unlock
* @param lockKey
* @param lockValue
* @param threadName
*/
@Override
public synchronized void unlock(String lockKey, long lockValue, String threadName) {
LOG.info(threadName + " Perform the unlock ==========");// Normal direct deletion If the lock is turned off abnormally, the expiration time will be determined
// To obtain redis Set the time in
String result = redisTemplate.opsForValue().get(lockKey);
Long currt_lock_timeout_str = result ==null?null:Long.valueOf(result);
// If the lock is added, the lock is removed, If not, wait for automatic expiration and recompete for locks
if (currt_lock_timeout_str !=null && currt_lock_timeout_str == lockValue){
redisTemplate.delete(lockKey);
LOG.info(threadName + " unlocked ------------------");
}
}
/**
* Multi-server clusters, using the following method, instead System.currentTimeMillis() To obtain redis Time to avoid multiple service times not 1 Problem!!
* @return
*/
@Override
public long currtTimeForRedis(){
return redisTemplate.execute(new RedisCallback<Long>() {
@Override
public Long doInRedis(RedisConnection redisConnection) throws DataAccessException {
return redisConnection.time();
}
});
}
}
3. Distributed lock validation
@RestController
@RequestMapping("/distribution/redis")
public class RedisLockController {
private static final String LOCK_NO = "redis_distribution_lock_no_";
private static int i = 0;
private ExecutorService service;
@Autowired
private StringRedisTemplate redisTemplate;
/**
* simulation 1000 Two threads execute the business and modify the resource simultaneously
*
* Thread pools are defined 20 A thread
*
*/
@GetMapping("lock1")
public void testRedisDistributionLock1(){
service = Executors.newFixedThreadPool(20);
for (int i=0;i<1000;i++){
service.execute(new Runnable() {
@Override
public void run() {
task(Thread.currentThread().getName());
}
});
}
}
@GetMapping("/{key}")
public String getValue(@PathVariable("key") String key){
Serializable result = redisTemplate.opsForValue().get(key);
return result.toString();
}
private void task(String name) {
// System.out.println(name + " Task in progress "+(i++));
// create 1 a redis A distributed lock
RedisLockImpl redisLock = new RedisLockImpl(redisTemplate);
// Lock time
Long lockTime;
if ((lockTime = redisLock.lock((LOCK_NO+1)+"", name))!=null){
// Start the mission
System.out.println(name + " Task in progress "+(i++));
// Mission accomplished Close the lock
redisLock.unlock((LOCK_NO+1)+"", lockTime, name);
}
}
}
4. Result verification:
In Controller, 1000 threads are simulated, submitted by thread pool, and 20 threads at a time preempt the distributed lock, grab the execution code of the distributed lock, and wait for those that do not
The results are as follows:
2017-04-07 16:27:17.385 INFO 8652 --- [pool-2-thread-4] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-4 Waiting to be locked, sleep 100 ms
2017-04-07 16:27:17.385 INFO 8652 --- [pool-2-thread-7] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-7 unlocked ------------------
2017-04-07 16:27:17.391 INFO 8652 --- [pool-2-thread-5] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-5 Locking success ++++ 111111
pool-2-thread-5 Task in progress 994
2017-04-07 16:27:17.391 INFO 8652 --- [pool-2-thread-5] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-5 Perform the unlock ==========
2017-04-07 16:27:17.391 INFO 8652 --- [pool-2-thread-1] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-1 Waiting to be locked, sleep 100 ms
2017-04-07 16:27:17.391 INFO 8652 --- [pool-2-thread-5] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-5 unlocked ------------------
2017-04-07 16:27:17.397 INFO 8652 --- [pool-2-thread-6] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-6 Locking success ++++ 111111
pool-2-thread-6 Task in progress 995
2017-04-07 16:27:17.398 INFO 8652 --- [pool-2-thread-6] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-6 Perform the unlock ==========
2017-04-07 16:27:17.398 INFO 8652 --- [pool-2-thread-6] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-6 unlocked ------------------
2017-04-07 16:27:17.400 INFO 8652 --- [ool-2-thread-19] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-19 Locking success ++++ 111111
pool-2-thread-19 Task in progress 996
2017-04-07 16:27:17.400 INFO 8652 --- [ool-2-thread-19] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-19 Perform the unlock ==========
2017-04-07 16:27:17.400 INFO 8652 --- [ool-2-thread-19] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-19 unlocked ------------------
2017-04-07 16:27:17.571 INFO 8652 --- [ool-2-thread-11] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-11 Locking success ++++ 111111
pool-2-thread-11 Task in progress 997
2017-04-07 16:27:17.572 INFO 8652 --- [ool-2-thread-11] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-11 Perform the unlock ==========
2017-04-07 16:27:17.572 INFO 8652 --- [ool-2-thread-11] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-11 unlocked ------------------
2017-04-07 16:27:17.585 INFO 8652 --- [pool-2-thread-4] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-4 Locking success ++++ 111111
pool-2-thread-4 Task in progress 998
2017-04-07 16:27:17.586 INFO 8652 --- [pool-2-thread-4] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-4 Perform the unlock ==========
2017-04-07 16:27:17.586 INFO 8652 --- [pool-2-thread-4] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-4 unlocked ------------------
2017-04-07 16:27:17.591 INFO 8652 --- [pool-2-thread-1] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-1 Locking success ++++ 111111
pool-2-thread-1 Task in progress 999
2017-04-07 16:27:17.591 INFO 8652 --- [pool-2-thread-1] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-1 Perform the unlock ==========
2017-04-07 16:27:17.591 INFO 8652 --- [pool-2-thread-1] c.jay.service.redis.impl.RedisLockImpl : pool-2-thread-1 unlocked ------------------
Thank you for reading, I hope to help you, thank you for your support of this site!