A solution to the Redis high concurrency problem

This article describes a solution to the high concurrency problem of Redis. To share for your reference, specific as follows:

Why does redis have high concurrency problems

redis's birth

redis is a single thread mechanism of nosql database, based on key-value, data can be persistent disk. redis itself has no concept of locking because it is single-threaded, and multiple client connections do not compete, but problems arise when concurrent access to redis is accessed by clients such as jedis. Connection timeouts, data conversion errors, blocking, client closure, and other issues occurred due to client connection confusion.

Also, the single-threaded nature of high concurrency dictates that operations on the same key will be queued for processing, and if the amount of concurrency is high, it may cause subsequent request timeouts.
When remote access to redis, due to network and other reasons caused high concurrent access delay return problem.

The solution

1. Pool the connection on the client side, and use the internal lock synchronized to read and write Redis on the client side.

2. From the perspective of server, setnx directional realization of locking mechanism is adopted. I don't know how this method would work in the real world.

jedis Common error analysis

Exception code 1:

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool

Problem analysis: redis.clients.util.Pool.getResource One available redis connection is returned from the JedisPool pool. Several important parameters regarding the configuration of available connections in JedisPool are as follows:

1.MaxActive: The maximum number of connection instances available. There is no limit when the number is negative.

2.MaxIdle: The maximum number of idle connection instances, with no limit when negative.

3.MaxWait: Timeout for waiting to get a link.

In other words, when there is no connection from active/idle in the connection pool, the maxWait time will be waited and the exception of Could not get a resource the pool will be thrown if there is no connection available in the wait timeout. So to avoid this kind of mistake,
We should reasonably set the values of these three parameters according to the actual situation of the program. At the same time, we should also reasonably handle this exception in the program method where we get a connection. When no connection is available, it may be a better choice to wait for a period of time to get the connection.

Exception code 2:

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out

If you encounter this exception, it may be confusing to know that redis is a memory operation with a speed of one millisecond, and that it is confusing to know that redis operates in seconds, but as I said at the beginning of this article, it is not surprising that redis runs out of time in some special cases. jedis should pass according to the actual situation when initializing JedisPool redis.clients.jedis.JedisPoolConfig Set the connection pool parameters reasonably and set the timeout for socket to read the input InputStream through the redisPool constructor.

`pool = new JedisPool(config, host, port, 100000)`;

The fourth parameter is time out in milliseconds. You can avoid this problem by setting this value reasonably. But that doesn't completely solve the problem of timeouts. In some cases of high concurrency, the delay return time can be as high as 10 seconds. The problem is solved at the code level by redis single-thread itself, which does not support locking, and which competes for concurrent operations on the same key.