Of Stack

Handling of Redis connection timeout exceptions

  • 2020-10-23 21:17:35
  • OfStack

0. Problem description

Using Jedis to connect redis for data query operation, the normal code operation is no problem, but the following errors are reported from time to time:

[

Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.util.RedisInputStream.read(RedisInputStream.java:181)
at redis.clients.jedis.Protocol.processBulkReply(Protocol.java:181)
at redis.clients.jedis.Protocol.process(Protocol.java:155)
at redis.clients.jedis.Protocol.processMultiBulkReply(Protocol.java:206)
at redis.clients.jedis.Protocol.process(Protocol.java:157)
at redis.clients.jedis.Protocol.processMultiBulkReply(Protocol.java:206)
at redis.clients.jedis.Protocol.process(Protocol.java:157)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:285)
at redis.clients.jedis.Connection.getObjectMultiBulkReply(Connection.java:291)
at redis.clients.jedis.BinaryJedis.hscan(BinaryJedis.java:3390)
at com.ict.mcg.filter.DuplicateClueFilterV2.hscan(DuplicateClueFilterV2.java:867)
at com.ict.mcg.filter.DuplicateClueFilterV2.collectRecentCluekeywords(DuplicateClueFilterV2.java:487)
at com.ict.mcg.main.GetCluesMain.run_online(GetCluesMain.java:208)
at com.ict.mcg.main.GetCluesMain.main(GetCluesMain.java:1685)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
... 16 more

]

The reason can be located as ES62en. net. SocketTimeoutException: Read timed out, namely abnormal network connection;

1. Possible causes

1.1 Server resources include high utilization of memory, disk, cpu, etc

After checking the status information of the redis deployment machine, it was found that the whole machine was running well

1.2 The server set up a firewall, resulting in the connection failure

Because the normal code flow can run through, so the firewall setting is no problem;

1.3 redis configuration file bind Monitoring host is improperly configured

The configuration of bind corresponding to host in the configuration file of redis is as follows: 


# By default Redis listens for connections from all the network interfaces
# available on the server. It is possible to listen to just one or multiple
# interfaces using the "bind" configuration directive, followed by one or
# more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1

The default bind binding of host is 0.0.0.0, that is, every available network interface can be monitored. Equivalent to configuration as: 


bind 0.0.0.0

Our configuration files are also configured correctly, and the normal code flow is running correctly, which can also prove this point;

1.4 Configuration of Jedis usage

The current connection pool configuration of Jedis is as follows: 


private static JedisPool getPool() {
    if (pool == null) {
      JedisPoolConfig config = new JedisPoolConfig();
      // control 1 a pool How many can be allocated jedis Instance, through pool.getResource() To obtain; 
      // If the assignment is zero -1 , means no restriction; if pool Has been allocated maxActive a jedis Instance, then pool The status of exhausted( Run out of ) . 
      config.setMaxActive(10);
      // control 1 a pool How many states can I have at most idle( free ) the jedis Instance. 
      config.setMaxIdle(2);
      // Said when borrow( The introduction of )1 a jedis At instance, the maximum wait time, and if the wait time is exceeded, it is thrown directly JedisConnectionException ; 
      config.setMaxWait(1000 * 200000);
      // in borrow1 a jedis Instance, whether to proceed ahead of time validate Operation; If it is true , you get jedis Examples are available; 
      config.setTestOnBorrow(true);
      config.setTestOnReturn(true);

      // At present redis only 1 A server 
      pool = new JedisPool(config, "localhost", 6379);
    }
    return pool;
  }

  private static Jedis getJedis() {
    Jedis jedis = null;
    int count = 0;
    do {
      try {
        pool = getPool();
        jedis = pool.getResource();
      } catch(Exception e) {
//    System.out.println(e.getMessage());
        e.printStackTrace();
        pool.returnBrokenResource(jedis);
      }

      count++;
    } while (jedis==null && count < 3);

    return jedis;
  }

In the logic of building JedisPool, only ES109en. setMaxWait(1000 * 200000) is set; , which is the maximum wait time for the introduction of the new jedis instance, without any other related connection timeout configuration; Looking at the JedisPool source code, I find the following: 


public JedisPool(final Config poolConfig, final String host) {
    this(poolConfig, host, Protocol.DEFAULT_PORT, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(String host, int port) {
    this(new Config(), host, port, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final String host) {
    this(host, Protocol.DEFAULT_PORT);
  }

  public JedisPool(final Config poolConfig, final String host, int port,
      int timeout, final String password) {
    this(poolConfig, host, port, timeout, password, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, final int port) {
    this(poolConfig, host, port, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, final int port, final int timeout) {
    this(poolConfig, host, port, timeout, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, int port, int timeout, final String password,
          final int database) {
    super(poolConfig, new JedisFactory(host, port, timeout, password, database));
  }

As can be seen from the above code, JedisPool has multiple overloaded constructors, and the constructor needs to pass 1 timeout parameter as the connection timeout time, if there is no transmission, Protocol.DEFAULT_ES120en as the default timeout time, continue to trace the source code: 


public final class Protocol {

  public static final int DEFAULT_PORT = 6379;
  public static final int DEFAULT_TIMEOUT = 2000;
  public static final int DEFAULT_DATABASE = 0;

  public static final String CHARSET = "UTF-8";

  public static final byte DOLLAR_BYTE = '$';
  public static final byte ASTERISK_BYTE = '*';
  public static final byte PLUS_BYTE = '+';
  public static final byte MINUS_BYTE = '-';
  public static final byte COLON_BYTE = ':';

  private Protocol() {
 // this prevent the class from instantiation
  }

Can conclude that the default JedisPool connection timeout time to 2 seconds, by default, and we call JedisPool constructor, just use this configuration, as long as two seconds without the connection is successful, redis connection is disconnected, and an error, it is in the database request have a larger concurrency value as possible, then do the following changes, at the time of creating JedisPool, into a larger timeout: 


pool = new JedisPool(config, ParamUtil.REDIS_ADDRESS[0], ParamUtil.REDIS_PORT, 1000 * 10);

2, summarize

Encounter problems or more check, look at the source code, look at the configuration of the source code, carefully item by item troubleshooting problems!

Related articles: