A single nginx 504 Gateway Time out error check resolve the record

Remember 1 time inexplicable site lost response troubleshooting. Previously, website 1 used nginx as the proxy back-end apache to run php to provide services. Often, apache will appear irregularly and for an indefinite period of time. The service will not be unresponsive, and then nginx will appear "504 Gateway Time-out ".
Looking at the error log, you can't see anything, assuming it's bug for apache (it's not, and we'll see why below).

Maybe when people get older, they don't like to toss about, so they are willing to keep the apache intact, use monitoring tools, and restart apache after receiving the alarm. Finally, I got bored for one day, which is to deal with php. I don't need apache head office anymore. In my anger, I used the source to install php-fpm and transferred to php-fpm to run php. Installing php is not a hassle, and the source installation works just fine. All you need to do is set the php worker worker process's log to output the php error log.

When the cut is ready, replace the original proxy_pass with fastcgipass.

upstream apachephp  {
    server www.ofstack.com:8080; #Apache1
}
....
proxy_pass  http://apachephp;

Replace into

upstream php {
        server 127.0.0.1:9000;
}
...
fastcgi_pass php;

You can move php from apache to php-fpm.
I thought I'd be able to rest easy, and it would be fine if you did, but if you didn't analyze the root cause of the problem. Problems would still come to me. On the second day, nginx reported gateway timeout for 504. This time, apache has no business. apache has distanced himself.

That should still be on nginx and php-fpm, check the error log for nginx, you can see it

[error] 6695#0: *168438 upstream timed out (110: Connection timed out) while reading response header from upstream,
...
request: "GET /kd/open.php?company=chinapost&number=PA24977020344 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.ofstack.com"

"GET /kd/ open.php "GET /kd/ open.php? company=chinapost&number=PA24977020344 HTTP/1.1" timeout exits.

Restart php-fpm immediately, the problem is gone, the website is accessible.

When I visited this page again, there was still no response, but it was normal to visit other pages at the same time. After the page was refreshed several times, the whole website was bad gateway timeout.

The problem narrowed down to this php script.

netstat -napo |grep "php5-fpm" | wc -l

Check that the php worker process has reached the upper limit of 10 in the configuration file, and there is a feeling that everyone is stuck in the open.php script.

What does this script do? This script is to collect express information, which USES php_curl.

The PHP script will force an exit if the execution time exceeds the configuration item max_execution_time in php.ini with no results.

Check that max_execution_time in php.ini is indeed matched, with a value of 30.

The universal google comes in handy. After google you get the following sentence

The set_time_limit() function and the configuration instruction max_execution_time only affect the execution time of the script itself. The maximum execution time of any script that occurs such as system call using system(), stream operation, database operation, etc., is not included when the script has been run.

If you don't set a timeout, php will wait for the result of the call.

Check the open.php source file 1. Sure enough, there is no timeout for curl.

Add the following two lines, refresh, and the problem is solved.

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10); //timeout on connect
curl_setopt($ch, CURLOPT_TIMEOUT, 10); //timeout on response

Of course, in addition to this method, php-fpm also provides a parameter that we can force to kill a long time fruitless process, except that this parameter is not turned on by default.

The configuration file of php-fpm can set a parameter request_terminate_timeout, the timeout of request termination, when the request is executed beyond this time will be kill.

It also has a parameter, request_slowlog_timeout, for logging slow requests.

You can use this code to run php from the command line

$real_execution_time_limit = 60; // The time limit 
if (pcntl_fork())
{
// some long time code which should be
// terminated after $real_execution_time_limit seconds passed if it's not
// finished by that time
}
else
{
sleep($real_execution_time_limit);
posix_kill(posix_getppid(), SIGKILL);
}