Detailed Explanation of php Multi process Application Scenario Example
- 2021-12-12 08:22:00
- OfStack
In this paper, an example is given to describe the php multi-process application scenario. Share it for your reference, as follows:
Introduction to pcntl
Extended introduction
The php multi-process module relies on the pcntl extension. The official manual introduces: http://php.net/manual/zh/book.pcntl.php
Note:
1. This extension is not available on the Windows platform.
2. Process control cannot be applied in Web server environment, and may bring unexpected results when it is used in Web service environment. Therefore, multiple processes can no longer be used in PHP Web development.
Installation extension
# Pass pecl Installation pcntl Expand
sudo pecl install pcntl
# Increase extension=pcntl.so
sodo vim /etc/php.ini
# Check whether the extension is installed successfully
php -m | grep pcntl
Processing files
When a file contains many tasks (1 line for each task) and there is no order of execution among the tasks, the file can be divided (the number of divided files is 1 to the number of processes), and then multi-processes are used for processing.
For example, there are now 10 mailbox accounts stored in the file mailist. txt, and it takes 2s to send messages each time, but it takes 20% to send these messages in turn in a single process.
If multiple processes, such as three processes, are used for processing, the file needs to be split into three small files according to the number of lines, in which two files are four records and one file is two records. If each process processes a small file, the time spent by different processes to send mail is 8, 8, 6, and the maximum total time spent is 8s.
Split a file
Original file maillist. txt
000000@163.com
111111@163.com
222222@163.com
333333@163.com
444444@163.com
555555@163.com
666666@163.com
777777@163.com
888888@163.com
999999@163.com
Split operation
split -a 1 -l 4 maillist.txt task
Split file
taska
000000@163.com
111111@163.com
222222@163.com
333333@163.com
taskb
444444@163.com
555555@163.com
666666@163.com
777777@163.com
taskc
888888@163.com
999999@163.com
Related scripts
Multi-process call script text_task. php
$cmds = [
['/Users/zhezhao/www/work/text_mail.php','a'],
['/Users/zhezhao/www/work/text_mail.php','b'],
['/Users/zhezhao/www/work/text_mail.php','c']
];
foreach ($cmds as $cmd){
$pid = pcntl_fork();
if($pid == -1){
exit('create process failed');
}
if($pid > 0){
pcntl_wait($status,WNOHANG);
}else{
pcntl_exec('/usr/bin/php',$cmd);
}
}
Multi-process execution script text_mail. php
require 'MailWork.php';
$name = $argv[1];
echo $name." start #".time().PHP_EOL;
$worker = new MailWork($name);
$res = $worker->text_mail($argc,$argv);
if($res === false){
echo $worker->getLastError();
}else{
echo $name." ".$res." works done # ".time().PHP_EOL;
}
Output result
c start #1504499765
b start #1504499765
a start #1504499765
b#mailto:444444@163.com
c#mailto:888888@163.com
a#mailto:000000@163.com
b#mailto:555555@163.com
a#mailto:111111@163.com
c#mailto:999999@163.com
c 2 works done # 1504499769
a#mailto:222222@163.com
b#mailto:666666@163.com
b#mailto:777777@163.com
a#mailto:333333@163.com
b 4 works done # 1504499773
a 4 works done # 1504499773
Three processes (a, b, c) were created in text_task. php, with four records in the files processed by a and b, and two records in the files processed by c.
From the output results, you can get:
1. a, b and c are executed at the same time, with the start timestamp of 1504499765
2. c is completed first, with a completion timestamp of 1504499769 and a time consuming of 4s
3. a and c are completed at the same time, with a completion timestamp of 1504499773 and a time consuming of 8s
Handling Message Queuing
This is another common one, where we put time-consuming operations into the message queue, fetch and execute records from the message queue through scripts. If a single process reads and processes messages in turn, it is easy to accumulate a large amount of data in the queue, resulting in a long delay time. In this scenario, multiple processes can read and process messages. The pop operation in redis is atomic, and there is no case where multiple queues are read to the same message.
Multi-process call script redis_task. php
$redis = new Redis();
$redis->connect('192.168.1.10');
$task_key = 'task_list';
$task_list = [
'000000@163.com',
'111111@163.com',
'222222@163.com',
'333333@163.com',
'444444@163.com',
'555555@163.com',
'666666@163.com',
'777777@163.com',
'888888@163.com',
'999999@163.com',
];
foreach ($task_list as $task){
$redis->lPush($task_key,$task);
}
$cmds = [
['/Users/zhezhao/www/work/redis_mail.php','a'],
['/Users/zhezhao/www/work/redis_mail.php','b'],
['/Users/zhezhao/www/work/redis_mail.php','c']
];
foreach ($cmds as $cmd){
$pid = pcntl_fork();
if($pid == -1){
exit('create process failed');
}
if($pid > 0){
pcntl_wait($status,WNOHANG);
}else{
pcntl_exec('/usr/bin/php',$cmd);
}
}
Multi-process execution script redis_mail. php
require 'MailWork.php';
$name = $argv[1];
echo $name." start #".time().PHP_EOL;
$worker = new MailWork($name);
$redis = new Redis();
$redis->connect('192.168.1.10');
$task_key = 'task_list';
while($redis->lLen($task_key)>0){
$mailto = $redis->rPop($task_key);
$worker->redis_mail($mailto);
}
echo $name." work done # ".time().PHP_EOL;
Output result
b start #1504499844
c start #1504499844
a start #1504499844
b#mailto:000000@163.com
a#mailto:111111@163.com
c#mailto:222222@163.com
b#mailto:333333@163.com
a#mailto:444444@163.com
c#mailto:555555@163.com
b#mailto:666666@163.com
a#mailto:777777@163.com
c#mailto:888888@163.com
c work done # 1504499850
a work done # 1504499850
b#mailto:999999@163.com
b work done # 1504499852
From the output results, you can get
1. a, b and c are executed at the same time with the timestamp of 1504499844
2. The execution of a and c was finished at the same time, and three records were processed respectively, with the timestamp of 1504499850 and the time consuming of 6s
3. b finally finished execution and processed 4 records with timestamp 1504499852, which took 8s
master-worker Mode
We simulate how the Web server handles http requests, creating a process for each request to process the content of the request.
class WebServer
{
private $list;
public function __construct()
{
$this->list = [];
}
public function worker($request){
$pid = pcntl_fork();
if($pid == -1){
return false;
}
if($pid > 0){
return $pid;
}
if($pid == 0){
$time = $request[0];
$method = $request[1];
$start = time();
echo getmypid()."\t start ".$method."\tat".$start.PHP_EOL;
sleep($time);
$end = time();
$cost = $end-$start;
echo getmypid()."\t stop \t".$method."\tat:".$end."\tcost:".$cost.PHP_EOL;
exit(0);
}
}
public function master($requests){
$start = time();
echo "All request handle stop at ".$start.PHP_EOL;
foreach ($requests as $request){
$pid = $this->worker($request);
if(!$pid){
echo 'handle fail!'.PHP_EOL;
return;
}
array_push($this->list,$pid);
}
while(count($this->list)>0){
foreach ($this->list as $k=>$pid){
$res = pcntl_waitpid($pid,$status,WNOHANG);
if($res == -1 || $res > 0){
unset($this->list[$k]);
}
}
usleep(100);
}
$end = time();
$cost = $end - $start;
echo "All request handle stop at ".$end."\t cost:".$cost.PHP_EOL;
}
}
$requests = [
[1,'GET index.php'],
[2,'GET index.php'],
[3,'GET index.php'],
[4,'GET index.php'],
[5,'GET index.php'],
[6,'GET index.php']
];
$server = new WebServer();
$server->master($requests);
Output:
All request handle stop at 1504513048
18945 start GET index.php at1504513048
18944 start GET index.php at1504513048
18946 start GET index.php at1504513048
18947 start GET index.php at1504513048
18948 start GET index.php at1504513048
18949 start GET index.php at1504513048
18944 stop GET index.php at:1504513049 cost:1
18945 stop GET index.php at:1504513050 cost:2
18946 stop GET index.php at:1504513051 cost:3
18947 stop GET index.php at:1504513052 cost:4
18948 stop GET index.php at:1504513053 cost:5
18949 stop GET index.php at:1504513054 cost:6
All request handle stop at 1504513054 cost:6
If the requests are processed in turn, the total time consumption is 1 +2 +3 +4 +5 +6 = 21s. Each request creates 1 process processing mode, and the total time consuming is the most time consuming request operation 6s.
Multi-processes are best used separately in methods, functions, or files, so that the logic is clearer and easier to analyze and maintain.
Appendix
Mail Action Class:
MailWork.php
<?php
/**
* Created by PhpStorm.
* User: zhezhao
* Date: 2017/9/4
* Time: Morning 10:05
*/
class MailWork
{
private $error;
private $name;
public function __construct($name)
{
$this->name = $name;
}
public function getLastError(){
return $this->error;
}
public function checkEnv($argc)
{
if (substr(php_sapi_name(), 0, 3) !== 'cli') {
$this->error ="This Programe can only be run in CLI mode";
return false;
}
if($argc!=2){
$this->error = 'wrong params';
return false;
}
return true;
}
public function getFileName($argv){
$filename = "task".$argv[1];
if(!file_exists($filename)){
$this->error = 'file does not exits';
return false;
}else{
return $filename;
}
}
public function sendMail($mailto)
{
sleep(2);
echo $this->name."#mailto:".$mailto.PHP_EOL;
}
public function text_mail($argc,$argv){
if(!$this->checkEnv($argc)){
return false;
}
$filename = $this->getFileName($argv);
if(!$filename){
return false;
}
$fp = fopen($filename,'r');
$counter = 0;
while(!feof($fp)){
$line = fgets($fp);
$mailto = substr($line,0,strlen($line)-1);
if(preg_match('/^\w+@\w+\.\w+$/',$mailto)){
$this->sendMail($mailto);
$counter++;
}
}
return $counter;
}
public function redis_mail($mailto){
if(preg_match('/^\w+@\w+\.\w+$/',$mailto)){
$this->sendMail($mailto);
}
}
}
For more readers interested in PHP related content, please check the topics on this site: Summary of PHP Process and Thread Operation Skills, Summary of PHP Network Programming Skills, Introduction to PHP Basic Syntax, Encyclopedia of PHP Array (Array) Operation Skills, Summary of php String (string) Usage, Introduction to php+mysql Database Operation Skills and Summary of php Common Database Operation Skills
I hope this article is helpful to everyone's PHP programming.