Redis tutorial command execution process details

preface

I've written a series of articles that have explored Redis's data structure, database implementation, key's expiration policy, and how Redis handles events. So we are only one step away from a stand-alone implementation of Redis, which is how Redis handles the commands sent by client and returns the results, so let's take a closer look at how Redis executes the commands under 1.

Read this article and you will learn:

How does Redis execute commands sent from remote clients

Redis client (Client)

Redis is a single-threaded application. How does it link to multiple client resume networks and process commands?
Because Redis is based on I/O multiplexing, in order to be able to handle multiple client requests, Redis locally creates an redisClient data structure for every client linked to the Redis server, which contains each client's individual state and commands executed. The Redis server USES one linked list to maintain multiple redisClient data structures.

On the server side, a linked list is used to manage all redisClient.

struct redisServer {
 //...
 list *clients;  /* List of active clients */
 //...
}

So I looked at the data structures and important parameters that redisClient contains:

typedef struct redisClient {

 //  Client status flag 
 int flags;  /* REDIS_SLAVE | REDIS_MONITOR | REDIS_MULTI ... */
 
 //  Socket descriptor 
 int fd;

 //  The database currently in use 
 redisDb *db;

 //  Of the database currently in use  id  (number) 
 int dictid;

 //  Client name 
 robj *name;  /* As set by CLIENT SETNAME */

 //  Query buffer 
 sds querybuf;

 //  Query buffer length peak 
 size_t querybuf_peak; /* Recent (100ms or more) peak of querybuf size */

 //  The number of arguments 
 int argc;

 //  Parameter object array 
 robj **argv;

 //  Records the commands executed by the client 
 struct redisCommand *cmd, *lastcmd;

 //  Type of request: Inline or multiple commands 
 int reqtype;

 //  The number of remaining unread command contents 
 int multibulklen; /* number of multi bulk arguments left to read */

 //  The length of the command content 
 long bulklen;  /* length of bulk argument in multi bulk request */

 //  Reply to the list 
 list *reply;

 //  Returns the total size of the objects in the list 
 unsigned long reply_bytes; /* Tot bytes of objects in reply list */

 //  Bytes sent, processed  short write  with 
 int sentlen;  /* Amount of bytes already sent in the current
    buffer or object being sent. */

 //  Return offset 
 int bufpos;
 //  Recovery buffer 
 char buf[REDIS_REPLY_CHUNK_BYTES];
 // ...
}

It is important to note here that redisClient does not refer to the remote client, but to the local data structure of the 1 Redis service, which we understand to be a mapping or proxy for the remote client.

flags

flags represents the role of the current client and its current state. It's special that it can represent one state alone or more states.

querybuf

querybuf is an sds dynamic string type. The buf specification is that it is just a buffer for storing commands that are not parsed.

argc & argv

The above querybuf is an unprocessed command. When Redis parses the querybuf command, the number and parameters obtained will be saved in argc and argv respectively. argv is an array of 1 redisObject.

cmd

Redis saved all redisCommand using one dictionary. key is the name of redisCommand, and the value is 1 redisCommand structure. This structure holds the implementation function of the command, the flag of the command, the number of arguments that the command should be given, the execution times of the command, the total elapsed time and other statistical information. cmd is 1 redisCommand.

When Redis resolves argv and argc, it will query the corresponding redisCommand in the dictionary according to the array argv[0]. In the example above, Redis will look up redisCommand corresponding to the command SET in the dictionary. redis executes the implementation functions of the commands in redisCommand.

buf & bufpos & reply

buf is an array of length REDIS_REPLY_CHUNK_BYTES. After Redis performs the corresponding operation, the returned data will be stored in buf. bufpos is used to record the number of bytes used in buf. When the data to be recovered is greater than REDIS_REPLY_CHUNK_BYTES, redis will use the linked list of reply to store the data.

The other parameters

The other parameters you can see by looking at the comments, literally. The omitted parameters basically relate to the Redis cluster management parameters, which will be covered in future articles.

Client connection and disconnection

As mentioned above, redisServer USES one linked list to maintain all redisClient states. Every time a client initiates a link, one corresponding redisClient data structure will be generated in Redis and added to the linked list of clients.

A client is likely to be disconnected for a variety of reasons.

There are several types:

Client actively exits or is kill. timeout timeout. Redis to protect itself, will break the development of data over the limit size of the client. To protect itself, Redis interrupts clients that need to return more data than the limit size.

Call summary

When the nested words on the client and server sides become readable, the server invokes the command request handler to do the following:

Reads the data in the nested words and writes to querybuf. Parse the commands in querybuf and record them in argc and argv. Find the corresponding recommand according to argv[0]. Execute the implementation functions corresponding to recommand. After execution, the results are stored in buf & bufpos & In reply, back to the caller.

Redis Server (server)

The above article observed the execution of commands from the perspective of redisClient. The following part of this article will observe how Redis implements the execution of commands from the perspective of Redis code.

redisServer startup

Before you understand how redisServer works, you need to understand what the startup of redisServer does:

You can continue to observe the main() function of Redis.

int main(int argc, char **argv) {
 //...
 //  Creates and initializes the server data structure 
 initServer();
 //...
}

We'll focus on the function initServer(), which initializes the server's data structure. Continue tracking code:

void initServer() {
 //...
 // create eventLoop
 server.el = aeCreateEventLoop(server.maxclients+REDIS_EVENTLOOP_FDSET_INCR);
 /* Create an event handler for accepting new connections in TCP and Unix
 * domain sockets. */
 //  for  TCP  Connection associated connection response ( accept ) processor 
 //  Used to receive and reply to clients  connect()  call 
 for (j = 0; j < server.ipfd_count; j++) {
 if (aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE,
  acceptTcpHandler,NULL) == AE_ERR)
  {
  redisPanic(
   "Unrecoverable error creating server.ipfd file event.");
  }
 }

 //  Associate the reply handler for the local socket 
 if (server.sofd > 0 && aeCreateFileEvent(server.el,server.sofd,AE_READABLE,
 acceptUnixHandler,NULL) == AE_ERR) redisPanic("Unrecoverable error creating server.sofd file event.");
 //...
}

Due to space limitations, we have omitted a lot of code irrelevant to this article and kept the core logic code.

In the previous article, The Event-driven Model in Redis, we explained that redis USES different event handlers to handle different events.

In this code:

eventLoop for the event handler was initialized Two event handlers, acceptTcpHandler and acceptUnixHandler, are registered with eventLoop to handle remote and local links, respectively.

The creation of redisClient

The acceptTcpHandler event handler is triggered when a remote client connects to the Redis server.

The acceptTcpHandler event handler creates a link. Then proceed to call acceptCommonHandler.

The acceptCommonHandler event handler is used to:

Call the createClient() method to create redisClient Check that the amount of redisClient that has been created exceeds the maximum number of server allowed If the limit is exceeded, the remote connection is denied Otherwise create redisClient created successfully And updates the count of connections, updating the flags field of redisClinet

At this time, Redis creates the redisClient data structure on the server side, and at this time, the remote client creates a proxy in redisServer. The remote client then establishes contact with the Redis server and sends a command to the server.

Processing command

In createClient() row count:

//  Bind reads events to events  loop  (Start receiving command requests) 
if (aeCreateFileEvent(server.el,fd,AE_READABLE,readQueryFromClient, c) == AE_ERR)

readQueryFromClient is registered with eventLoop. The purpose of readQueryFromClient is to read the client's query buffer contents from client.

The function processInputBuffer is then called to handle the client request. There are several core functions in processInputBuffer:

processInlineBuffer and processMultibulkBuffer parse the commands in querybuf and record them in argc and argv. processCommand finds the corresponding recommen according to argv[0], and executes the corresponding execution function of recommend. The correctness of the command is also verified before execution. Save the results to buf & bufpos & In the reply

Return the data

When you're done, you need to return the data to the remote caller. The call chain is as follows

processCommand - > addReply - > prepareClientToWrite

In prepareClientToWrite we have the familiar code:

aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,sendReplyToClient, c) == AE_ERR) return REDIS_ERR;

The sendReplyToClient event handler is bound to eventloop.

By observing the code in sendReplyToClient, it is found that if bufpos is greater than 0, buf will be sent to the remote client; if the length of reply is greater than 0, reply will be traversed and sent to the remote client. It should be noted here that in order to avoid the excessive amount of DATA in reply, Redis will be excessively occupied and slow accordingly. To solve this problem, when the total number of writes is greater than REDIS_MAX_WRITE_PER_EVENT, Redis temporarily interrupts the write, records the progress of the operation, gives the processing time to other operations, and the rest is left to continue next time. This routine we have seen too many road.

conclusion

When the remote client connects to redis, the redis server creates an redisClient proxy for the remote client. redis reads the data in the nested words and writes to querybuf. Parse the commands in querybuf and record them in argc and argv. Find the corresponding recommand according to argv[0]. Execute the corresponding execution function of recommend. After execution, save the results to buf & bufpos & reply. Returns to the caller. When the data is returned, the size of the written data will be controlled. Guarantee the corresponding time of redis.

As a single-threaded application, Redis follows the idea that each step has an upper limit (including the upper limit of execution time or the upper limit of file size). Once the upper limit is reached, the current progress will be recorded and executed next time. This ensures that Redis can respond in a timely manner without blocking.