Discussion on Nginx seven layer reverse agent and load balancing
- 2020-05-15 03:31:46
- OfStack
1. Introduction
1.1 Nginx is not only an excellent web software, but also an excellent 7-tier agent and load balancing software. Nginx ACTS as a front-end agent. When users request services, they can judge according to url and then allocate to different background webserver.
1.2 load balancing principle of Nginx: firstly, web server pool in http module is configured using upstream module to define proxy-web. In the pool, we can add multiple background webserver, in which the status check and scheduling algorithm are configured in the pool. The virtual host is then defined in the serverr module, but this virtual host does not specify its own web directory site. Instead, it will match url with location and forward it to the web pool defined above, and then forward it to web server in the background according to the scheduling policy
2. Introduction of load balancing configuration items
2.1 introduction to upstream scheduling algorithm
(1) rr polling (default)
If RS goes down, it will be automatically deleted. By default, only port 80 will be detected. If RS reports 402, 403, 503, 504 errors, it will be directly returned to the client.
(2) weight (weight)
On the basis of rr, add the weight (the default is rr+weight), the weight polling is proportional to the access, the larger the value is, the more the allocation is, you can set the weight according to the server configuration, which can solve the problem of uneven server performance for request allocation
(3) the ip_hash
Solve the sharing problem of dynamic web page session
Each access request is allocated according to the hash value of the IP address. As long as the hash value of ip is the same, it will be allocated to the same server (the -p parameter of lvs load balancing, persistence_timeout 50 in the configuration of keepalived). This scheduling algorithm can solve the sharing problem of session dynamic web page, but sometimes it will lead to uneven distribution of requests.
Tip: since nat mode is used in China, hash is not suitable for use
ip_hash cannot be used with other algorithm 1 block, that is, cannot make weight or backup
(4) fair (third party)
According to the response time of the back-end server to configure, the priority allocation of short response time is more intelligent than the above. This algorithm can intelligently carry out load balancing according to the page size and load time. nginx itself does not support fair, so you need to download upstrea_fair module of nginx
(5) url_hash (third party)
Mainly used on cache servers
The request is allocated according to the accessed url, and the same url is directed to the same server. The effect is more significant when the back-end server is the cache server. hash statement is added in upstream, and weight and other parameters cannot be written in server statement.
Cons: if one machine goes down, it's a pain, and consistent_hash can fix it
It can improve the efficiency of the back-end cache server. nginx itself does not support url_hash, so you need to download hash software
(6) least_conn
Minimum number of connections, which fewer connections are allocated to which device
(7) consistent_hash
1 assignment algorithm
2.2 introduction to upstream health check configuration
upstream proxy_nginx {
server 192.168.0.254 weight=1 max_fails=2 fail_timeout=10s ;
server 192.168.0.253 weight=2 max_fails=2 fail_timeout=10s;
server 192.168.0.252 backup ;
server 192.168.0.251 down ;
}
server 192.168.0.254: background RS, can be domain name or IP, default is port 80, can also be added :80 specified
wight = 1 weight ratio defaults to 1
max_fails=2 the maximum number of failed health checks, exceeding which means that RS is not available, and 1,0 by default means that failed attempts are prohibited. Production environment 1 is normally set 2~3 times
fail_timeout=10s failed timeout, default is 10s
The hot standby configuration of backup automatically starts when all the RS in the front are unavailable
down says the service will never be available
Note: the lower the setting of max_fails is, the better the user experience will be. However, there is a disadvantage if the setting is too low, that is, proxy may misjudge the state of RS, and the lower the setting of RS is, the greater the chance of misjudgment will be, which will have a huge impact on the business. When the number of RS is relatively small, it is suggested to set the value to a larger point.
2.3 introduction to the use of the location directive
url Location are mainly used for matching, such as: http: / / www beyond. com/nice, here for location www. beyond. com is a domain name, is/nice url.
For url matches, you can use a string or a regular expression, but if it is a regular expression, you must specify the prefix, location, to match different url, and then apply a different configuration if the match is successful
Syntax: location [=|~|~*|^~|@]/url {... . }
[=] exact match, if match equals is found, stop the search immediately, and process the request immediately (highest priority)
[~] means matching a regular expression, case - sensitive
[^~] matches only strings, does not match regular expressions, and is mainly used to match directories
[~*] means matching a regular expression, case - insensitive
[@] specify a named location, which generally applies only to internal redirection requests, location @name {· · · ·}
Example:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name www.beyond.com;
#No.1
location / {
return 502;
}
#No.3
location ~* \.jpg$ {
return 403;
}
#No.4
location ^~ /a/ {
return 402;
}
#No.5
location /a/1.jpg {
return 401;
}
#No.6
location = /a/.jpg {
return 400;
}
}
}Summary:
Matching order:
1) match normal url first, but continue to match the regular after the match
The internal matching rule is prefixed with the maximum matching, that is, location /data/123 {} takes precedence over location /data {}, regardless of the order in which location is placed. After the match, the maximum match prefix is used to continue to match the following regular match
2) regular matching,
Regular matching has nothing to do with the order of position, but with the order of logic. The maximum match will prevail (the more accurate the better).
3) the execution logic of location basically has nothing to do with the order; However, for the method of regular matching, if the first url is matched, the next url will not be matched.
In this case, if normal localtion is matched and there is no regular match, normal match is used. If there is both a maximum prefix match for normal location and a regular match, the regular match overrides the maximum prefix match.
4) after matching "normal location", sometimes it is necessary to continue matching "regular location", and sometimes it is not necessary to continue matching "regular location".
In both cases, there is no need to continue regular location matching :(1) when normal location is preceded by "^~", Nginx is specifically told to continue regular location matching when normal location 1 is matched; (2) when normal location happens to match strictly, not the maximum prefix match, the regular match will not continue.
2.4 introduction of proxy module
The Nginx forwarding module is ngx_http_proxy_module, which is installed by default and can be used directly
Proxy_pass this directive is used to forward url matched by location to serve pool.
Syntax: proxy_pass URL;
Note: please note whether URL contains URI when using this directive. If URL does not contain URI, nginx will not change the uri of the original address. If uri is included, the new uri will be used instead of the original uri.
Such as:
location /test {
#1 proxy_pass http://1.1.1.1;
#2 proxy_pass http://1.1.1.1/tmp;
}
If use 1, original url will not change, after the agent is http: / / 1.1.1.1 / test
If you use 2, original url will become http: / / 1.1.1.1 / tmp
Therefore, when configuring url, you need to pay attention to the question whether to add "/" at the end of url
Parameters:
proxy_ignore_client_abort on|off is used to set whether nginx interrupts the request to the proxy server when the client interrupts the network request. The default is off, interrupt
proxy_headers_hash_max_size size sets the size of the hash table of the http header to 512 by default
proxy_headers_hash_bucket_size size sets the unit size of the hash table capacity for which the http header is applied. The default is 64 characters
The client_body_buffer_size client requests the cache size, which can be understood to save the local before sending it to the user
The timeout time for proxy_connect_timeout time and RS links, by default, is 60s
proxy_send_timeout time RS
The time proxy_read_timeout time waits for RS to respond, indicating that the connection has been successful and that the queue is in progress
Whether proxy_buffering on|off is enabled or not proxy buffer is on by default
The size of the proxy_buffer_size cache is, by default, equal to the size set by the directive proxy_buffers to 4K or 8K
The number and size of the proxy_buffers number size buffer, the response information obtained from RS is placed into the buffer, which defaults to 8 4K|8K
proxy_busy_buffers_size the size of proxy_buffers that can be used when the system is busy, the official recommended size is twice that of proxy_buffers, and the default is 8K or 16K
proxy_temp_path [level1[level2]] specifies a file path on disk to temporarily store the bulk response data of the proxy server. If the buffer is full but the response data is still not fully received by nginx, the response data will be temporarily stored in this file
proxy_max_temp_file_size is used to configure the total size of all temporary files
proxy_temp_file_write_size is used to configure the amount of data to be written to the cache temporary file at the same time
proxy_set_header host $host needs to be specified one by one when RS has multiple virtual hosts
proxy_set_header X-Forwarded-For $remote_addr opens the load balancer to forward the IP address of a real customer to RS.
Note:
"proxy_set_header" when our RS has multiple virtual hosts (the same ip, the same port), such as www, bbs, blog, how will the proxy server know where to send the request? At this time, nginx agent will look up the proxy_set_header parameter, send the request to the corresponding domain name of the virtual host.
3. Load balancing configuration case of nginx
1. There are three domain names: www.beyond.com bbs.beyond.com film.beyond.com, and four webserver are responsible for providing the service
2. There is one mailserver and one cloud server on one server
Planning:
web1 192.168.254.251 BBS, film
web2 192.168.254.252 BBS, film
web3 192.168.254.253 BBS, film
web4 192.168.254.254 www, load balancer
web5 192.168.254.250 mail cloud
Note: web4 is the load balancer and is also responsible for parsing the www site. web1, web2, web3 are responsible for parsing the bbs and film sites. Mail and cloud are on the same device and also on the web site
Nginx installation, slightly has the need to can see this tutorial https: / / www ofstack. com article / 128758. htm
The nginx configuration file for web4 is now listed for reference
nginx.conf :
user nginx;
worker_processes 4;
error_log logs/error.log;
pid logs/nginx.pid;
events {
use epoll ;
worker_connections 1024;
http {
include mime.types;
default_type application/octet-stream;
server_tokens off;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
client_header_buffer_size 32k;
large_client_header_buffers 4 128k;
client_max_body_size 10m;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 8k;
gzip_http_version 1.1;
gzip_comp_level 6;
gzip_vary on;
gzip_types text/javascript text/plain application/x-javascript text/css application/xml;
log_format main ' $http_host $http_x_forwarded_for ${request_time}s [$time_local] "$request" $status $body_bytes_sent $http_referer $http_user_agent $remote_addr';
include vhosts/*.conf; # Per virtual host 1 Configuration files
include upstream.conf; # store web The background of server
fastcgi_intercept_errors on;
}
Vhosts/{bbs,film,www,cloud,mail}.conf :
server {
listen 80;
server_name bbs.beyond.com;
index index.php;
location / {
proxy_pass http://web;
proxy_set_header host $host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 4K;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4K;
proxy_buffers 4 32K;
proxy_busy_buffers_size 64K;
proxy_temp_file_write_size 64K;
}
}
server {
listen 80;
server_name cloud.beyond.com;
index index.php;
location / {
proxy_pass http://192.168.254.250:8000;
proxy_set_header host $host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 4K;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4K;
proxy_buffers 4 32K;
proxy_busy_buffers_size 64K;
proxy_temp_file_write_size 64K;
}
}
server {
listen 80;
server_name film.beyond.com;
index index.php;
location / {
proxy_pass http://web;
proxy_set_header host $host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 4K;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4K;
proxy_buffers 4 32K;
proxy_busy_buffers_size 64K;
proxy_temp_file_write_size 64K;
}
}
server {
listen 80;
server_name mail.beyond.com;
index index.php;
location / {
proxy_pass http://192.168.254.250:80;
proxy_set_header host $host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 4K;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4K;
proxy_buffers 4 32K;
proxy_busy_buffers_size 64K;
proxy_temp_file_write_size 64K;
}
}
server {
listen 80;
server_name monitor.beyond.com;
index index.php;
location / {
proxy_pass http://192.168.254.220;
proxy_set_header host $host;
proxy_set_header X-Forwarded-For $remote_addr;
client_body_buffer_size 4K;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4K;
proxy_buffers 4 32K;
proxy_busy_buffers_size 64K;
proxy_temp_file_write_size 64K;
}
}
server {
listen 80;
server_name www.beyond.com;
root /usr/local/nginx/html/www;
index index.html;
access_log logs/current/www.beyond.com-access.log main;
error_log logs/current/www.beyond.com-error.log;
error_page 500 501 502 503 504 /error/5-error.html;
error_page 400 403 404 405 408 410 411 412 413 414 415 /error/4-error.html;
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ {
expires 30d;
}
location ~.*\.(js|css|javascript|fluash)$ {
expires 24h;
}
location /error {
root /usr/local/nginx/logs/error;
}
}
Note that each server is a.conf file
upstream. conf file:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name www.beyond.com;
#No.1
location / {
return 502;
}
#No.3
location ~* \.jpg$ {
return 403;
}
#No.4
location ^~ /a/ {
return 402;
}
#No.5
location /a/1.jpg {
return 401;
}
#No.6
location = /a/.jpg {
return 400;
}
}
}