Everyone should know the Redis expiration key and expiration policy

Today, I would like to share with you a piece of Redis's content about expired keys. There are mainly four contents:

How do I set the expiration key How do I unset the expiration time What is the expiration policy for expired keys What about the handling of expired keys by RDB, AOF, and replication

Sets the lifetime or expiration time of the key

redis 1 has four commands to set the lifetime of the key (how long it can survive) or expiration time (when it is deleted)

expire < key > < ttl > : Sets the key survival time to ttl seconds pexpire < key > < ttl > : Sets the survival time of key to ttl milliseconds expireat < key > < timestamp > : Sets the expiration time of key to the number of seconds timestamp specified by timestamp pexpireat < key > < ttl > : Sets the expiration time of key to the number of milliseconds timestamp specified by timestamp

All four of the above commands are essentially implemented through the pexpireat command.

 Example: 
127.0.0.1:6379> set a test
OK
127.0.0.1:6379> EXPIRE a 5
(integer) 1
127.0.0.1:6379> get a //  Distance to set the survival time command  5  Seconds to perform 
"test"
127.0.0.1:6379> get a //  Distance to set the survival time command  5  Seconds after implementation 
(nil)

127.0.0.1:6379> set b 12
OK
127.0.0.1:6379> EXPIREAT b 1545569500
(integer) 1
127.0.0.1:6379> time
1) "1545569486"
2) "108616"
127.0.0.1:6379> get b //  Distance setting  1545569500  Executes within the specified number of seconds timestamp 
"12"
127.0.0.1:6379> time
1) "1545569506"
2) "208567"
127.0.0.1:6379> get b //  Distance setting  1545569500  Executes after the specified number of seconds timestamp 
(nil)

If we accidentally set the expiration time wrong, we can delete the previous expiration time

Remove expiration time

persist < key > The command removes the expiration time of a key, for example:

127.0.0.1:6379> EXPIRE c 1000
(integer) 1
127.0.0.1:6379> ttl c //  Have an expiration date 
(integer) 9996
127.0.0.1:6379> PERSIST c
(integer) 1
127.0.0.1:6379> ttl c //  No expiration time 
(integer) -1

PS : ttl  In seconds, the remaining survival time of the return key; In the same way and  pttl  The command returns the remaining lifetime of the key in milliseconds 

At this point, if we didn't remove the expiration time, then if a key expires, when will it be deleted?

There are three answers to this question, each representing three different deletion strategies

Deletion policy for expired keys

Time to delete

While setting the key expiration time, create a timer that immediately deletes the key when the key expiration time comes.

Pros: Most memory-friendly. You can release the memory occupied by the key in time.

Cons: Unfriendly to CPU. Especially in the case of too many expired keys, deleting the expired key will take up a considerable part of CPU time. At the same time, using CPU to delete expired keys that the current task does not want to close when memory is not tight and CPU is tight will undoubtedly have an impact on the server response time and throughput.

Lazy to delete

Keys are left to expire, but each time a key is read or written from a key space, the retrieved key is checked for expiration. Delete the delete if it expires, or return the key. (PS: Keyspace is a data structure that holds all key-value pairs in the database)

Pros: Most CPU friendly. Expiration is checked only at the time of the operation, the target of deletion is limited to the current key that needs to be processed, and no CPU time is spent deleting any other expired keys that are irrelevant to the operation.

Cons: Memory unfriendly. This 10 is easy to understand, the key is out of date, but because 1 is not accessed, 1 is kept (unless you manually perform the flushdb operation to empty all key in the current database). , equivalent to a memory leak.

Periodically delete

Every once in a while, the program checks the database and deletes the expired keys. The algorithm determines how many expired keys to delete and how many databases to check.

This policy is a compromise between the above two policies, and needs to set the execution time and frequency of the delete operation according to the actual situation.

Once you understand the strategy for deleting expired keys, what strategy does the redis server use to delete expired keys?

In fact, the Redis server USES both lazy deletion and periodic deletion strategies, and by working together, the server is able to balance CPU and memory.

Lazy deletion is the redis server built-in policy. Periodic deletion can be set in the following two ways:

Configure hz option of ES114en. conf, default is 10 (that is, 10 executions per second, the higher the value, the faster the refresh frequency, and the higher the performance cost of Redis) Configure the maximum maxmemory for ES118en. conf to trigger an active cleanup policy when the maxmemory limit has been exceeded

RDB's handling of expired keys

Generate the RDB file

The program is checked by keys in the database, and expired keys are not saved to the newly created RDB file. Therefore, expired keys in the database do not affect the generation of new RDB files

Load the RDB file

Here we need to explain the situation separately:

If the server is running in primary server mode, the program checks the keys saved in the RDB file when it loads, and the expired keys are not loaded into the database. So the expired key does not affect the primary server on which the RDB file was loaded. If the server is running in slave mode, when the RDB file is loaded, it will be loaded into the database regardless of whether the key is expired. But because the master-slave server is doing data synchronization, the slave server's data is emptied. Therefore, in general, the expired key does not affect the slave servers that load RDB files.

AOF's handling of expired keys

AOF file write

When the server is running in AOF persistent mode, if an expired key in the database has not been deleted, the AOF file will not be affected by the expired key and will remain.

When the expired key is deleted, the program appends an DEL command to the AOF file to explicitly record that the key was deleted.

AOF rewrite

When performing an AOF rewrite, it is also checked by the database's keys, and expired keys are not saved to the AOF file after the rewrite. Therefore, AOF rewrite will not be affected

Copy the processing of expired keys

When the server is running in replication mode, the primary server controls the deletion of expired key actions from the slave server in order to ensure that the primary and slave server data is 1 neutral.

How exactly does that work?

After the primary server removes an expired key, an DEL command is sent to all slave servers telling the slave server to remove the expired key After receiving the command from the server, delete the expired key

PS: When the slave server receives the client's read command for the expired key, it still returns the value corresponding to the key to the client without deleting it.

conclusion