The shell script is Shared as a daemon instance that ensures that the PHP script does not hang

  • 2020-07-21 07:19:00
  • OfStack

1 a few days ago began to run the data list, the list will need to provide the user name, whether there is a phone number, if there is a mailbox, user list I easily get into the, however, there are as many as 2000 w user list, and to determine whether the user has a mobile phone number, if there is a mailbox must through 1 open to the public security interface 1 a user to request, and then analysis the return value to know.

Here is my solution:
1. Save 2000w list to temporary data table
2. Use PHP program to obtain 500 users from the table each time, and generate the original records of SQL update after detection
3. In order to prevent the abrupt break of PHP program, the shell script is used to detect it every 1 minute. If PHP hangs, restart it
The reason why I use shell script as the daemon is that the detection interface between mobile phone and mailbox is slow, so it is impossible to detect 2000w users in 1-2 days.

Scheme Details:
1. Temporarily save users, the structure of which is as follows:


CREATE TABLE `users` ( 
  `account` varchar(50) COMMENT ' The user name ', 
  `has_phone` tinyint(3) unsigned NOT NULL default '0' COMMENT ' Do you have a cell phone number ', 
  `has_email` tinyint(3) unsigned NOT NULL default '0' COMMENT ' Is there an email address? ', 
  `flag` tinyint(3) unsigned  NOT NULL default '0' COMMENT ' Sign a ', 
  PRIMARY KEY  (`account`), 
  KEY `flag` (`flag`) 
 ) ENGINE=InnoDB  DEFAULT CHARSET=utf8 COMMENT=' List of tables '; 

I first imported more than 2000 w usernames into this temporary table. The has_phone and has_email fields are 0 by default (no), and the flag flag indicates whether the user has been detected.
The following is part 1 of the table data:
9873aaa,0,0,0
adddwwwd876222,0,0,0
testalexlee,0,0,0
codejia.net,0,0,0
haohdouywaa21,0,0,0

2. PHP script check_ES45en.php
The user list to import to the table, and then write a simple PHP script, thinking is this: each loop from the table flag = 0 500 users, and then request the interface to determine whether a user has a mobile phone number, email and generate 1 SQL, saved to a SQLS array, such as 500 users all after the test, cyclic SQLS array, the 500 list update list, and will be flag sign bit is set to 1, said it had finish testing, don't get next time.
Since the PHP script code is long, here is a simple code description:

<?php 
class Users{ 
    private $data; 
    private $sqls; 
    private $nums;         // Determine if there is 500 The user  
    private $total_nums;   // The current number of users that have been detected  

    // Every time I take 500 A user  
    private function getUsers(){...} 

    // To detect the 500 Five users and generate SQL 
    private function checkUserInfo(){...} 

    // To update this 500 A user  
    private function updateUserInfo(){...} 

    // run  
    public function run(){ 
        $flag = true; 
        while($flag){ 
             if($this->nums != 500){ $flag = false; } 
             if($this->total_nums == 10000){  
                exit(0); // Run the 1w The user exits and the daemon starts  
             } 
             $this->getUsers(); 
             $this->checkUserInfo(); 
             $this->updateUserInfo(); 
             sleep(1); // Run the 500 Users have a rest 1 Seconds to protect the user detection interface       
         } 
    } 
} 

$user = new Users(); 
$user->run(); 
?>

The above is a concise version of the PHP script. The initial version did not have the $total_nums variable, because when I started running this script, I found that I only finished running 4w multiple scripts and then hung the ball. Later, I saw that script 1 was suspended because the database was not connected. Adding this variable doesn't solve the problem, but after 1w users, the PHP script exits and is restarted by the shell script below.

3. shell script as daemon
I added this shell script to crontab, which is executed once every 1 minute. This shell script is very simple. It detects whether es73EN_ES74en. php process id exists. If it does not exist, it means that PHP script has finished running 1w (0) and the user quits, then the shell script starts the script and enters the detection of the next 1w user list.
As I mentioned above, if the PHP script is unable to connect to the database, PHP 1 will hang the ball there and cannot exit. I added a time detection to the shell script. When the PHP script process exists, I calculate how long it has existed. If it exceeds the time I expected, I will drop the PHP script kill and restart it.

The results of the sample data at the beginning are similar to the following:
testalexlee,1,0,1
codejia.net,0,0,1
haohdouywaa21,1,1,1
9873aaa,0,1,1
adddwwwd876222,1,0,1

In the end: the above user list data is just an example, don't be too serious, 2000w data, I estimate to run for a period of time, because the detection interface is relatively slow, interface after receiving a request to connect the table, look up the table, and then return. In fact, the best way is to pull a list directly from the interface request, and then use the shell command processing under the results soon, but in the company is like this, some things are not open, you know ~~~


Related articles: