Details the configuration function of Nginx to read the request body
- 2020-05-10 23:28:50
- OfStack
nginx core itself will not take the initiative to read the request body, the work is to request processing phase of the module to do, but nginx core provides ngx_http_read_client_request_body () interface to read the request body, it also provides a discard request body interface - ngx_http_discard_request_body (), at the different stages of the request execution, any one phase of the module if interested in the request body, or hope to lose a client sending a request body, You can do this by calling each of these interfaces separately. These two interfaces are the standard interfaces for handling request body provided by nginx core. If you want some instructions related to request body in the configuration file (such as client_body_in_file_only, client_body_buffer_size, etc.) to be expected to work, and if you want to use nginx built-in variables related to request body (such as $request_body and $request_body_file), 1 generally all modules must call these interfaces to perform the corresponding operations, and if a custom interface is required to handle the request body, try to be compatible with nginx's default behavior.
1. Read the body of the request
Request body read 1 a in nginx content handler, 1 nginx built-in modules, such as proxy module, fastcgi module, uwsgi module, these modules from the behavior must be a client request body (if any) to the corresponding protocol complete forwarded to the back-end service process, all these modules are invoked the ngx_http_read_client_request_body read () interface to complete the request body. It is important to note that these modules will read the request body of the client completely before forwarding the data to the back end.
Because of the limitation of memory ngx_http_read_client_request_body () interface experience part or all of the read request into a temporary file, according to the size of the request body and relevant configuration of instructions, request body may complete placed in 1 block of contiguous memory might be placed in two different memory, also may be all there is a temporary file, the last May 1 part in memory, the rest in a temporary file. The following instructions are related to these different storage behaviors:
client_body_buffer_size: set the size of the cache request body to buffer, which is twice the size of the system page by default. When the size of the request body exceeds this size, nginx will write the request body to a temporary file. The appropriate size can be set according to the business requirements to avoid io operation;
client_body_in_single_buffer: indicates whether the request body is completely stored in a block of continuous memory, the default is off. If this instruction is set to on, nginx will ensure that the request body is stored in a block of continuous memory when it is not larger than the value set by client_body_buffer_size, but when it exceeds the size, it will be written to a temporary file.
client_body_in_file_only: sets whether the request body is always saved in a temporary file, the default is off. When this specification is set to on, nginx creates a temporary file for the request even if the client display indicates that the request body length is 0.
The implementation of ngx_http_read_client_request_body() interface is then introduced, which is defined as follows:
ngx_int_t
ngx_http_read_client_request_body(ngx_http_request_t *r,
ngx_http_client_body_handler_pt post_handler) This interface has two parameters, the first is a pointer to the request structure, the second is a function pointer, when the request body is read, it will be called. As mentioned earlier, according to the existing behavior of nginx, the module logic will be executed after the request body is read. This callback function 1 is generally the logical processing function of the module. The ngx_http_read_client_body () function first adds 1 to the reference of the main request corresponding to the parameter r. The purpose of this is related to the context in which the interface is called. Generally speaking, the module calls this interface in content handler.
static ngx_int_t
ngx_http_proxy_handler(ngx_http_request_t *r)
{
...
rc = ngx_http_read_client_request_body(r, ngx_http_upstream_init);
if (rc >= NGX_HTTP_SPECIAL_RESPONSE) {
return rc;
}
return NGX_DONE;
}The above code calls ngx_http_request_request_body () function in porxy module content handler, ngx_http_proxy_handler(). ngx_http_upstream_init() is passed into the interface as a callback function. In addition, the context of content handler call of the module in nginx is as follows:
ngx_int_t
ngx_http_core_content_phase(ngx_http_request_t *r,
ngx_http_phase_handler_t *ph)
{
...
if (r->content_handler) {
r->write_event_handler = ngx_http_request_empty_handler;
ngx_http_finalize_request(r, r->content_handler(r));
return NGX_OK;
}
...
} In the above code, after content handler is called, its return value is used as a parameter to call ngx_http_finalize_request() function. When the request body is not fully received, ngx_http_client_request_body () function returns value of NGX_AGAIN, content handler, For example, ngx_http_handler () will return NGX_DONE, while NGX_DONE will be passed to ngx_http_finalize_request() as a parameter, which will cause the reference count of the primary request to be reduced by 1, so it will exactly offset the increment of the primary request count by 1 at the beginning of the function ngx_http_client_body ().
Then return to ngx_http_read_client_request_body () function, it will check whether the request of the request body has been read or discarded, and if so, is the callback function and return NGX_OK directly, here is actually to request inspection, request is a concept in nginx, nginx can be initiated in the current request and in one or more new request to access other location son, A detailed introduction to subrequests will be discussed in the following sections. In general, subrequests do not need to read the body of the request themselves.
The function then calls ngx_http_test_expect() to check if the client has sent Expect: 100-continue header, if yes, then reply "HTTP/1.1 100 Continue" to the client. According to the http 1.1 protocol, the client can send 1 Expect header to the server to indicate that it expects to send the request body. If the server allows the client to send the request body, it will reply "HTTP/1.1 100 Continue".
Continue the preparation for receiving the request body by assigning an ngx_http_request_body_t structure and saving it in r- > request_body, this structure is used to store cache references, temporary file references, remaining request body size and other information used in the request body reading process. It is defined as follows.
<span style="font-family:SimSun;font-size:18px;">typedef struct {
ngx_temp_file_t *temp_file;
ngx_chain_t *bufs;
ngx_buf_t *buf;
off_t rest;
ngx_chain_t *to_write;
ngx_http_client_body_handler_pt post_handler;
} ngx_http_request_body_t;</span>
After preparation, the function checks whether the request has an content_length header. If there is no header or the client sends an content_length header with a value of 0, it indicates that there is no body of the request. In this case, the callback function is directly called and NGX_OK is returned. Of course, if the client_body_in_file_only directive is set to on, and content_length is 0, the function will create an empty temporary file before calling the callback function.
Entering the lower half of the function indicates that the client request does indicate that the body of the request is to be sent, and the function first checks to see if the body of the request was read in advance of the request header, which is checked by determining the cache holding the request header (r-) > Is there any unprocessed data in header_in)? If there is pre-read data, assign an ngx_buf_t structure and place r- > The preread data in header_in is stored there, and if r- > There is space left in header_in to accommodate the remaining unread request bodies, which will continue to be used without allocating a new cache, and of course even if the request body has been preread in its entirety, there is no need to continue processing, at which point the callback function is called and returned.
If there is no preread data or if the preread is incomplete, this function allocates 1 new block of memory (unless r-) > If the request_body_single_buf instruction is set to no, the preread data will be copied into the newly developed memory block. The actual read request body is in the ngx_http_read_request_body () function, which loops the read request body and stores it in the cache. If the cache is full, the data is cleared and written back to the temporary file. Of course, it may not be possible to read the data once. This function will mount the read event and set the read event handler to ngx_http_client_body_handler. In addition, the core of nginx also sets the timeout between the two read events in the request body. nginx will return 408 to the client.
After the final reading of the request body, ngx_http_do_read_client_request_body() will adjust the request body to the desired location (memory or file) according to the configuration. In all cases, the request body can be transferred from r- > The request_body bufs linked list is obtained, which may have up to 2 nodes, each with 1 buffer, but the contents of this buffer may be stored in memory or in a disk file. In addition, the $request_body variable only gets the corresponding data when the request body has been read and is all saved in memory.
2. Discard the request body
1 module wants to take the initiative to discard the client sent a request body, you can call nginx core provides ngx_http_discard_request_body () interface, active discarded reason may have a lot of kinds, such as the business logic of the modules do not need to request body, the client sends the excessive request body, in addition to compatible http1. 1 agreement pipeline request, modules have a duty to take the initiative to throw away unwanted request body. In summary, in order to maintain good client compatibility, nginx must actively discard unwanted request bodies. Now start analyzing the ngx_http_discard_request_body() function:
ngx_int_t
ngx_http_discard_request_body(ngx_http_request_t *r)
{
ssize_t size;
ngx_event_t *rev;
if (r != r->main || r->discard_body) {
return NGX_OK;
}
if (ngx_http_test_expect(r) != NGX_OK) {
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
rev = r->connection->read;
ngx_log_debug0(NGX_LOG_DEBUG_HTTP, rev->log, 0, "http set discard body");
if (rev->timer_set) {
ngx_del_timer(rev);
}
if (r->headers_in.content_length_n <= 0 || r->request_body) {
return NGX_OK;
}
size = r->header_in->last - r->header_in->pos;
if (size) {
if (r->headers_in.content_length_n > size) {
r->header_in->pos += size;
r->headers_in.content_length_n -= size;
} else {
r->header_in->pos += (size_t) r->headers_in.content_length_n;
r->headers_in.content_length_n = 0;
return NGX_OK;
}
}
r->read_event_handler = ngx_http_discarded_request_body_handler;
if (ngx_handle_read_event(rev, 0) != NGX_OK) {
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
if (ngx_http_read_discarded_request_body(r) == NGX_OK) {
r->lingering_close = 0;
} else {
r->count++;
r->discard_body = 1;
}
return NGX_OK;
}
The function also checks the content-length header in the request header. If the client wants to send the body of the request, it must send the content-length header, and it also checks to see if the body of the request has been read elsewhere. If the request body is indeed to be processed, the function then checks the preread data in the request header buffer. The preread data is thrown away, but if the request body has been preread in its entirety, the function returns.
Next, if there are any remaining request bodies to be processed, the function calls ngx_handle_read_event() to mount the read event in the event handling mechanism and sets the read event handler to ngx_http_discarded_request_body_handler. With this in place, the function finally calls the ngx_http_read_discarded_request_body() interface to read the body of the request from the client and discard it. If the client does not send the request body once, the function returns, and the remaining data is sent to ngx_http_discarded_request_body_handler() for processing when the next read event arrives. In this case, the requested discard_body will be set to 1 to identify this situation. In addition, the number of references requested (count) is also added by 1. The purpose of this is that the client may not fully send the request body to be sent after nginx has processed the request. The addition of references is to prevent nginx core from directly releasing the resources related to the request after processing the request.
The ngx_http_read_discarded_body () function is very simple. It reads the data from the link and throws it away until it has read all the data in the receive buffer. If the request body has been read, the function sets the read event handler to ngx_http_block_reading.
Let's look at the handler function ngx_http_discarded_request_body_handler, which is called every time it reads an event. First, let's look at its source code:
void
ngx_http_discarded_request_body_handler(ngx_http_request_t *r)
{
...
c = r->connection;
rev = c->read;
if (rev->timedout) {
c->timedout = 1;
c->error = 1;
ngx_http_finalize_request(r, NGX_ERROR);
return;
}
if (r->lingering_time) {
timer = (ngx_msec_t) (r->lingering_time - ngx_time());
if (timer <= 0) {
r->discard_body = 0;
r->lingering_close = 0;
ngx_http_finalize_request(r, NGX_ERROR);
return;
}
} else {
timer = 0;
}
rc = ngx_http_read_discarded_request_body(r);
if (rc == NGX_OK) {
r->discard_body = 0;
r->lingering_close = 0;
ngx_http_finalize_request(r, NGX_DONE);
return;
}
/* rc == NGX_AGAIN */
if (ngx_handle_read_event(rev, 0) != NGX_OK) {
c->error = 1;
ngx_http_finalize_request(r, NGX_ERROR);
return;
}
if (timer) {
clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);
timer *= 1000;
if (timer > clcf->lingering_timeout) {
timer = clcf->lingering_timeout;
}
ngx_add_timer(rev, timer);
}
}
If the read event occurs before the request is processed, no timeout event is handled and no timer is set, and the function simply calls ngx_http_read_discarded_request_body() to read and discard the data.