Resolve the pits encountered in thread concurrent redisson use

Pit of thread concurrent redisson

Background

Because of one purchase demand in business, it is necessary to protect the inventory from oversold (we are not an e-commerce company). After investigation, we finally chose to use Redission for control.

Mainly because of Redission's rich API, open source framework, has been widely used in actual production environments.

Problem description

When we use the Lock. lock () method in Ression, if there is a common case of thread concurrency, the following exception occurs:

java.lang.IllegalMonitorStateException: attempt to unlock lock, not locked by current thread by node id: 9f178836-f7e1-44fe-a89d-2db52f399c0d thread-id: 22

Problem analysis

Before the end of thread-1, That is, when thread-1 acquires the lock but has not released it, ` thread-2 was stopped by another thread waiting to return from the blocking state of lock. tryLock to continue the following logic, and a runtime exception was thrown to try to release a lock belonging to thread thread-1, which caused the thread thread-2 to end. However, after thread-2 completed a series of operations, the thread thread-1 released its own lock.

So thread-2 does not acquire the lock, but executes what needs to be synchronized and tries to release the lock.

We will know the solution. The lock added by the current thread is unlocked by the current thread, that is to say, when we use lock. unlock, we can add the judgment of the thread.

Problem solving

 RLock lock = redissonClient.getLock(key);
    if(lock.isLocked()){ //  Is it still locked 
      if(lock.isHeldByCurrentThread()){ //  Is the lock of the current executing thread 
        lock.unlock(); //  Release lock 
      }
    }

Precautions for using redisson

Redisson is an Java in-memory data grid implemented on the basis of Redis. Compared with Jedis, which exposes the underlying operations, Redisson provides a series of distributed Java common objects and many distributed services.

Characteristic & Functions:

Support Redis Single Node (single) Mode, Sentinel (sentinel) Mode, Master-Slave (Master/Slave) Mode, and Cluster (Redis Cluster) Mode The program interface is called by asynchronous execution and asynchronous flow execution Data serialization, Redisson object coding class is used to serialize and deserialize the object, in order to achieve the object in Redis read and store Single collection data fragmentation. In cluster mode, Redisson provides automatic fragmentation for a single Redis collection type Provides a variety of distributed objects, such as Object Bucket, Bitset, AtomicLong, Bloom Filter, and HyperLogLog Provide rich distributed collections, such as Map, Multimap, Set, SortedSet, List, Deque, Queue, etc. Realization of distributed lock and synchronizer, reentrant lock (Reentrant Lock), fair lock (Fair Lock), interlock (MultiLock), red lock (Red Lock), semaphore (Semaphonre), expired signal lock (PermitExpirableSemaphore), etc Provides advanced distributed services such as distributed remote services (Remote Service), distributed real-time object (Live Object) services, distributed execution services (Executor Service), distributed scheduling task services (Schedule Service), and distributed mapping induction services (MapReduce) For more features and functions, please pay attention to official website: http://redisson.org

Implementation principle

redis itself does not support the above-mentioned distributed objects and collections. Redisson realizes advanced data structures and characteristics in the client by using the characteristics of redis, such as the realization of priority queue, which is sorted by the client and then stored in redis.

Client implementation means that when no client is online, all these data structures and features will not be preserved or automatically effective, such as the triggering of expired events or the addition of elements to the original priority queue.

Matters needing attention

Real-time

RMap has a function that can set the expiration time of key-value pairs and register event listeners for key-value pairs

Element elimination function (Eviction)

The distributed RMapCache Java object of Redisson implements the elimination mechanism for a single element based on RMap. While still preserving the insertion order of elements. Because RMapCache is implemented based on RMap, it inherits both java. util. concurrent. ConcurrentMap and java. util. Map interfaces. The integration of Spring and Cache provided by Redisson and JCache are based on this function.

The current Redis itself does not support element elimination in hashing (Hash), so all expired elements are regularly cleaned through org. redisson. EvictionScheduler instances. In order to ensure the effective use of resources, each run cleans up to 300 expired elements. The start time of the task will be automatically adjusted according to the actual number of cleanups last time, and the interval will be between 1 second and 1 hour. For example, if 300 elements are deleted during this cleanup, the next cleanup will be performed after 1 second (minimum interval). 1 Once the cleaning quantity is less than the last cleaning quantity, the time interval will increase by 1.5 times.

As stated in the official wiki, this function is cleaned regularly by background threads, so this is non-real-time (issue-1234: on expired event is not executed in real-time.), and the delay is between 5 seconds and 2 hours, so scenes with high real-time requirements have to be measured by themselves.

Due to the non-real-time nature of the expiration time, the occurrence of the expiration event is also non-real-time, and the corresponding listener may be delayed for a while before receiving the notification. In my test, the error of setting ttl at the second level is relatively large, while the minute level ttl is not bad (setting value on the left side, actual time consumption on the right side):

1s _ 5s
3s _ 5s
4s _ 5s
5s _ 9s
6s _ 10s
10s _ 15s
1m _ 1m11s

Serialization

The default encoder by Redisson is JsonJacksonCodec, and JsonJackson will have an infinite loop exception when serializing objects with bidirectional references. fastjson will automatically replace it with the reference $ref after checking out the bidirectional reference, ending the loop.

In my case, I serialized an service, which was hosted by spring and injected with another service. With fastjson, it can be serialized to redis normally, while JsonJackson throws an infinite loop exception.

In order to make the serialized content visible, we don't need other binary encoders of redission to realize the encoder by ourselves:

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.serializer.SerializerFeature;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.ByteBufInputStream;
import io.netty.buffer.ByteBufOutputStream;
import org.redisson.client.codec.BaseCodec;
import org.redisson.client.protocol.Decoder;
import org.redisson.client.protocol.Encoder;​
import java.io.IOException;​
public class FastjsonCodec extends BaseCodec {​
 private final Encoder encoder = in -> {
 ByteBuf out = ByteBufAllocator.DEFAULT.buffer();
 try {
 ByteBufOutputStream os = new ByteBufOutputStream(out);
 JSON.writeJSONString(os, in,SerializerFeature.WriteClassName);
 return os.buffer();
 } catch (IOException e) {
 out.release();
 throw e;
 } catch (Exception e) {
 out.release();
 throw new IOException(e);
 }
 };
​
 private final Decoder<Object> decoder = (buf, state) ->
 JSON.parseObject(new ByteBufInputStream(buf), Object.class);
​
 @Override
 public Decoder<Object> getValueDecoder() {
 return decoder;
 }
​
 @Override
 public Encoder getValueEncoder() {
 return encoder;
 }
}

Subscribe to Publication

Redisson's encapsulation of subscribed publications is RTopic, which is also the implementation principle of many event listening in Redisson (for example, event listening for key-value pairs).

When you use unit tests, you find that after an event is published, subscribers need to delay 1 time before receiving the event. Specific reasons remain to be investigated.