An example of PHP using CURL_MULTI to realize multithreaded acquisition

  • 2021-07-10 18:59:23
  • OfStack

In the past two days, a customer customized a login-free publishing module. Because many picture downloads are involved in the module, considering the performance problems, a function of CURL_MULTI remote collection of web pages is specially written to facilitate future use. It is estimated that the original single-threaded curl function will not be used to foreach in the future, and its performance comparison is obvious. Also get 10 different pages of my blog, curl_multi: 4.5246081352234, file_get_contents: 33.001797914505, nearly 8 times the efficiency. It is conceivable that if there are more attachments, the performance difference will be more obvious, hoping to help you!


<?php
$text = remote(array('https://www.ofstack.com/','http://www.baidu.com/'));
print_r($text); function remote($urls) {
    if (!is_array($urls) or count($urls) == 0) {
        return false;
    }     $curl = $text = array();
    $handle = curl_multi_init();
    foreach($urls as $k => $v) {
        $nurl[$k]= preg_replace('~([^:\/\.]+)~ei', "rawurlencode('\\1')", $v);
        $curl[$k] = curl_init($nurl[$k]);
        curl_setopt($curl[$k], CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($curl[$k], CURLOPT_HEADER, 0);
        curl_multi_add_handle ($handle, $curl[$k]);
    }     $active = null;
    do {
        $mrc = curl_multi_exec($handle, $active);
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);     while ($active && $mrc == CURLM_OK) {
        if (curl_multi_select($handle) != -1) {
            do {
                $mrc = curl_multi_exec($handle, $active);
            } while ($mrc == CURLM_CALL_MULTI_PERFORM);
        }
    }     foreach ($curl as $k => $v) {
        if (curl_error($curl[$k]) == "") {
        $text[$k] = (string) curl_multi_getcontent($curl[$k]);
        }
        curl_multi_remove_handle($handle, $curl[$k]);
        curl_close($curl[$k]);
    }
    curl_multi_close($handle);
    return $text;
}


Related articles: