An example of PHP using CURL_MULTI to realize multithreaded acquisition
- 2021-07-10 18:59:23
- OfStack
In the past two days, a customer customized a login-free publishing module. Because many picture downloads are involved in the module, considering the performance problems, a function of CURL_MULTI remote collection of web pages is specially written to facilitate future use. It is estimated that the original single-threaded curl function will not be used to foreach in the future, and its performance comparison is obvious. Also get 10 different pages of my blog, curl_multi: 4.5246081352234, file_get_contents: 33.001797914505, nearly 8 times the efficiency. It is conceivable that if there are more attachments, the performance difference will be more obvious, hoping to help you!
<?php
$text = remote(array('https://www.ofstack.com/','http://www.baidu.com/'));
print_r($text);
function remote($urls) {
if (!is_array($urls) or count($urls) == 0) {
return false;
}
$curl = $text = array();
$handle = curl_multi_init();
foreach($urls as $k => $v) {
$nurl[$k]= preg_replace('~([^:\/\.]+)~ei', "rawurlencode('\\1')", $v);
$curl[$k] = curl_init($nurl[$k]);
curl_setopt($curl[$k], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl[$k], CURLOPT_HEADER, 0);
curl_multi_add_handle ($handle, $curl[$k]);
}
$active = null;
do {
$mrc = curl_multi_exec($handle, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($handle) != -1) {
do {
$mrc = curl_multi_exec($handle, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
foreach ($curl as $k => $v) {
if (curl_error($curl[$k]) == "") {
$text[$k] = (string) curl_multi_getcontent($curl[$k]);
}
curl_multi_remove_handle($handle, $curl[$k]);
curl_close($curl[$k]);
}
curl_multi_close($handle);
return $text;
}