A detailed explanation of the timing task based on redis

preface

Requirements encountered in the business (under abstract Description 1) : The ability to loop tasks at different intervals for different users. For example, push relevant SMS messages to users 24 hours after successful registration.

Using crontab & # 63; It is too heavy, and almost unrealistic, to generate one timed task for every user on the server.
Regular polling ? IO is frequent and inefficient

Considering that redis, which is often used, can set the cache time, there should be expired event notification, I checked the documentation and found the relevant configuration, called "key-space event notification". For details, refer to the official documentation.

Technology stack

redis / nodeJs / koa

Key technical difficulties

Enable redis keyspace notification (only available in 2.8.0 and above) Use a separate redis db as much as possible redis-based distributed locks are used to ensure that related events are not consumed repeatedly The information that needs to be used twice needs to be reflected in key for the redis cache redis cache key USES a business prefix to avoid duplication of names Prevent business service restarts from invalidating listening at the nodejs level

"talk is cheap, show me the code 🤖"

The core code

 The core code 
const { saveClient, subClient } = require('./db/redis') //  The storage and subscription instances need to be two different instances 
const processor = require('./service/task')
const config = require('./config/index')
const innerDistributedLockKey = '&&__&&' //  Distributed locks used internally key The eigenvalues of the 
const innerDistributedLockKeyReg = new RegExp(`^${innerDistributedLockKey}`)

saveClient.on('ready', async () => {
 saveClient.config('SET', 'notify-keyspace-events', 'Ex') //  The storage instance is set to push key expiration events 
 console.log('redis init success')
})

subClient.on('ready', () => { //  All can still be initialized after the service is restarted processor
 subClient.subscribe(`__keyevent@${config.redis.sub.db}__:expired`) //  The subscription instance is responsible for subscribing messages 
 subClient.on('message', async (cahnnel, expiredKey) => {
  //  Distributed locked key Do not listen on processing 
  if (expiredKey.match(innerDistributedLockKeyReg)) return
  //  Simple distributed lock, get lock instance consumption event
  const cackeKey = `${innerDistributedLockKey}-${expiredKey}`
  const lock = await saveClient.set(cackeKey, 2, 'ex', 5, 'nx') //  The usage here allows for a simple distributed lock 
  if (lock === 'OK') {
   await saveClient.del(cackeKey)
   for (let key in processor) {
    processor[key](expiredKey) // processor This corresponds to the business logic executed after receiving the phase-critical expiration notification, such as pushing an SMS message, and then relating it processor Again in set1 It expires at a certain time key
   }
  }
 })
 console.log('subClient init success')
})

servide/task (processor)
exports.sendMessage = async function sendMessage(expiredKey, subClient) {
 //  Only the expiration event for the relevant business is handled 
 if (expiredKey.match(/^send_message/)) {
  const [prefix, userId, type] = expiredKey.split('-')
  let user = getUser(userId)
  if (user.phone) {
   push(message) //  Pseudo code 
   resetRedisKey(expiredKey, ttl) //  To put the key Set to 1 After a period of time expired, expired will trigger the logic again 
  }
 }
}

conclusion

This function USES the key space notification function of redis to realize simple timing task function based on user or different business scenarios. Because the key-space event notification function is an CPU intensive operation, it is recommended that a separate DB be used for processing. The basic usage is shown here. The persistence of timed tasks is not taken into account, and if redis fails to restart during use, all timed tasks will be lost. This event can also be lost if the subscription service fails and is not online at the time of the redis issue key invalidity notice, or if the network problem is not received by the consumer. The expired event for redis does not fire when key expires, but when key is deleted. redis periodically cleans up expired key or checks for expiration when visiting key. Only then does expired key trigger deletion, so there is a small time gap (the actual use of an individual does not affect the user experience).

Therefore, you need to balance the usage scenarios of timing tasks implemented using redis's expiration mechanism.