Do you know how many connections an Linux server can load

Preface

First, let's look at how to identify an TCP connection. The system is identified by a 4-tuple (src_ip, src_port, dst_ip, dst_port), namely source IP, source port, target IP and target port. For example, we have a service 192.168. 0.1 with Port 80 open. Then all clients will connect to Port 80 of this service. There is a misunderstanding, that is, we often say that a machine has 65,536 ports, so the number of connections it carries is 65,536. This statement is extremely wrong, which confuses the source port with the access target port. When we do pressure test, we use the pressure test client. The number of connections of this client is limited by the number of ports, but the number of connections on the server can reach thousands, and 1 can reach millions (4C8G configuration). As for the upper limit, it depends on the degree of optimization. The specific methods are as follows:

We are pressing a target server to see the number of connections under load. When we press to a fixed number, the console suddenly reports "too many open files", because when linux system creates an TCP connection, it will create an socket handle, and each socket handle is a file handle. The operating system has a limit on the number of open file handles. One of the basic philosophies of Unix/Linux is "all files are cut". To increase the carrying capacity of TCP, it is necessary to adjust the file handle.

Step 1: Modify the limit on the number of file handles

#  View the current user allowed TCP Maximum number of open file handles 
ulimit -n

#  Modify file handle 
vim /etc/security/limits.conf

* soft nofile 655350
* hard nofile 655350

After modification, exit the terminal window, log in again (no need to restart the server), and you can see the latest results. This is the first step of optimization, modifying the file handle limit.

Note:
soft nofile (soft limit) means that Linux limits the number of files that users can open at the same time by taking one step within the range that the current system can bear
hard nofile (hard limit) is the maximum number of files that the system can open at the same time calculated according to the system hardware resource condition (mainly system memory)
Soft constraints are usually less than or equal to hard constraints

Step 2: TCP parameter tuning

参数	默认配置	调整配置	说明
fs.file-max	1048576	9999999	所有进程打开的文件描述符数
fs.nr_open	1635590	1635590	单个进程可分配的最大文件数
net.core.rmem_default	124928	262144	默认的TCP读取缓冲区
net.core.wmem_default	124928	262144	默认的TCP发送缓冲区
net.core.rmem_max	124928	8388608	默认的TCP最大读取缓冲区
net.core.wmem_max	124928	8388608	默认的TCP最大发送缓冲区
net.ipv4.tcp_wmem	4096 16384 4194304	4096 16384 8388608	TCP发送缓冲区
net.ipv4.tcp_rmem	4096 87380 4194304	4096 87380 8388608	TCP读取缓冲区
net.ipv4.tcp_mem	384657 512877 769314	384657 512877 3057792	TCP内存大小
net.core.netdev_max_backlog	1000	5000	在每个网络接口接收数据包的速率比内核处理这些包的速率快时，允许送到队列的数据包的最大数目
net.core.optmem_max	20480	81920	每个套接字所允许的最大缓冲区的大小
net.core.somaxconn	128	2048	每1个端口最大的监听队列的长度，这是个全局的参数
net.ipv4.tcp_fin_timeout	60	30	对于本端断开的socket连接，TCP保持在FIN-WAIT-2状态的时间（秒）。对方可能会断开连接或1直不结束连接或不可预料的进程死亡
net.core.netdev_max_backlog	1000	10000	在每个网络接口接收数据包的速率比内核处理这些包的速率快时，允许送到队列的数据包的最大数目
net.ipv4.tcp_max_syn_backlog	1024	2048	对于还未获得对方确认的连接请求，可保存在队列中的最大数目。如果服务器经常出现过载，可以尝试增加这个数字
net.ipv4.tcp_max_tw_buckets	5000	5000	系统在同时所处理的最大timewait sockets数目
net.ipv4.tcp_tw_reuse	0	1	是否允许将TIME-WAIT sockets重新用于新的TCP连接
net.ipv4.tcp_keepalive_time	7200	900	表示TCP链接在多少秒之后没有数据报文传输时启动探测报文（发送空的报文）
net.ipv4.tcp_keepalive_intvl	75	30	表示前1个探测报文和后1个探测报文之间的时间间隔
net.ipv4.tcp_keepalive_probes	9	3	表示探测的次数

From the above configuration parameters, we can know that buffer queues are made for tcp sending and receiving in Linux kernel, which can improve the throughput of the system.

These parameters are defined in/etc/sysctl. conf file. Some parameters may not be defined in the file. The system has given default values. If you need to modify them, add or modify them directly in the file, and then execute sysctl-p command to make them take effect.

Note:
Parameter values are not set as large as possible, and some need to consider the hardware configuration of the server and the influence of parameters on other services on the server.

Summarize