Summary of the six ways PHP calls remote urls

  • 2020-03-31 16:45:49
  • OfStack

Example code 1: get the contents with file_get_contents

<?php 
$url='http://www.baidu.com/'; 
$html=file_get_contents($url); 
//print_r($http_response_header); 
ec($html); 
printhr(); 
printarr($http_response_header); 
printhr(); 
?> 


Example code 2: open the url with fopen and get the content

<? 
$fp=fopen($url,'r'); 
printarr(stream_get_meta_data($fp)); 
printhr(); 
while(!feof($fp)){ 
$result.=fgets($fp,1024); 
} 
echo"url body: $result"; 
printhr(); 
fclose($fp); 
?> 

Example code 3: get the url by post using the file_get_contents function

<?php 
$data=array('foo'=>'bar'); 
$data=http_build_query($data); 

$opts=array( 
'http'=>array( 
'method'=>'POST', 
'header'=>"Content-type: application/x-www-form-urlencodedrn". 
"Content-Length: ".strlen($data)."rn", 
'content'=>$data 
), 
); 
$context=stream_context_create($opts); 
$html=file_get_contents('http://localhost/e/admin/test.html',false,$context); 
echo$html; 
?> 

Example code 4: open the url with the fsockopen function and get the complete data, including the header and body

<? 
functionget_url($url,$cookie=false){ 
$url=parse_url($url); 
$query=$url[path]."?".$url[query]; 
ec("Query:".$query); 
$fp=fsockopen($url[host],$url[port]?$url[port]:80,$errno,$errstr,30); 
if(!$fp){ 
returnfalse; 
}else{ 
$request="GET$queryHTTP/1.1rn"; 
$request.="Host:$url[host]rn"; 
$request.="Connection: Closern"; 
if($cookie)$request.="Cookie: $cookien"; 
$request.="rn"; 
fwrite($fp,$request); 
while(!@feof($fp)){ 
$result.=@fgets($fp,1024); 
} 
fclose($fp); 
return$result; 
} 
} 
//Get the HTML part of the url, remove the header
functionGetUrlHTML($url,$cookie=false){ 
$rowdata=get_url($url,$cookie); 
if($rowdata) 
{ 
$body=stristr($rowdata,"rnrn"); 
$body=substr($body,4,strlen($body)); 
return$body; 
} 
returnfalse; 
} 
?> 

Example code 5: open the url with the fsockopen function and get the complete data, including the header and body, as a POST

<? 
functionHTTP_Post($URL,$data,$cookie,$referrer=""){ 
// parsing the given URL 
$URL_Info=parse_url($URL); 

// Building referrer 
if($referrer=="")// if not given use this script. as referrer 
$referrer="111"; 

// making string from $data 
foreach($dataas$key=>$value) 
$values[]="$key=".urlencode($value); 
$data_string=implode("&",$values); 

// Find out which port is needed - if not given use standard (=80) 
if(!isset($URL_Info["port"])) 
$URL_Info["port"]=80; 

// building POST-request: 
$request.="POST ".$URL_Info["path"]." HTTP/1.1n"; 
$request.="Host: ".$URL_Info["host"]."n"; 
$request.="Referer:$referern"; 
$request.="Content-type: application/x-www-form-urlencodedn"; 
$request.="Content-length: ".strlen($data_string)."n"; 
$request.="Connection: closen"; 
$request.="Cookie: $cookien"; 
$request.="n"; 
$request.=$data_string."n"; 

$fp=fsockopen($URL_Info["host"],$URL_Info["port"]); 
fputs($fp,$request); 
while(!feof($fp)){ 
$result.=fgets($fp,1024); 
} 
fclose($fp); 
return$result; 
} 
printhr(); 
?> 

Example code 6: use the curl library. Before using the curl library, you may want to check php.ini to see if the curl extension is turned on

<? 
$ch = curl_init(); 
$timeout = 5; 
curl_setopt ($ch, CURLOPT_URL, 'http://www.baidu.com/'); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout); 
$file_contents = curl_exec($ch); 
curl_close($ch); 
echo $file_contents; 
?> 

About the curl library:
Curl's official website is http://curl.haxx.se/
Curl is a FILE transfer tool that USES URL syntax and supports FTP, FTPS, HTTP HTPPS SCP SFTP, TFTP TELNET DICT FILE, and LDAP. Curl supports SSL certificates, HTTP POST, HTTP PUT, FTP upload, kerberos, upload based on HTT format, proxies, cookies, user + password proof, file transfer recovery, HTTP proxy channels, and a host of other useful tricks

<? 
functionprintarr(array$arr) 
{ 
echo"<br> Row field count: ".count($arr)."<br>"; 
foreach($arras$key=>$value) 
{ 
echo"$key=$value <br>"; 
} 
} 
?> 

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
PHP code to grab remote web site data
Now there may be many hobbyists will encounter the same question, is how to search like a search engine to grab other people's web site HTML code, and then collect the code into their own useful data! Let me give you some simple examples today.

An example of Ⅰ . Grab the remote page title:
Here is the code snippet:

<?php 
 

error_reporting(7); 
$file = fopen ("//www.jb51.net/", "r"); 
if (!$file) { 
echo "<font color=red>Unable to open remote file.</font>n"; 
exit; 
} 
while (!feof ($file)) { 
$line = fgets ($file, 1024); 
if (eregi ("<title>(.*)</title>", $line, $out)) { 
$title = $out[1]; 
echo "".$title.""; 
break; 
} 
} 
fclose($file); 

//End 
?> 

Examples of Ⅱ . Grab the remote web HTML code:

Here is the code snippet:

<? php 
 

$fp = fsockopen("www.dnsing.com", 80, $errno, $errstr, 30); 
if (!$fp) { 
echo "$errstr ($errno)<br/>n"; 
} else { 
$out = "GET / HTTP/1.1rn"; 
$out .= "Host:www.dnsing.comrn"; 
$out .= "Connection: Close rnrn"; 
fputs($fp, $out); 
while (!feof($fp)) { 
echo fgets($fp, 128); 
} 
fclose($fp); 
} 
//End 
?> 
The above two code snippets are directly Copy back to run to know the effect, the above example is just the prototype of grasping web data, to make it more suitable for their own use, there are different situations.

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

The functions that make a little sense are: get_content_by_socket(), get_url(), get_content_url(), and get_content_object.
< ? PHP

// get all the content urls and save them to a file
The function get_index ($save_file, $prefix = "index_") {
$count = 68;
$I = 1;
If (file_exists ($save_file)) @ unlink ($save_file);
$fp = fopen ($save_file, "a +") or die (" Open ". $save_file. The "failed");
While ($i< ${count)
$url = $prefix. $i. ".htm";
Echo "Get ". $url."..." ;
$url_str = get_content_url (get_url ($url));
Echo "OK \ n";
Fwrite ($fp, $url_str);
+ + $I;
}
Fclose ($fp);
}

// get the target multimedia object
The function get_object ($url_file, $save_file, $STR = "| - : * * : - |") {
If (! File_exists ($url_file)) die($url_file." not exist");
$file_arr = file ($url_file);
If (! Is_array ($file_arr) || empty($file_arr)) die($url_file." not content");
$url_arr = array_unique ($file_arr);
If (file_exists ($save_file)) @ unlink ($save_file);
$fp = fopen($save_file, "a+") or die("Open save file ". $save_file." failed");
The foreach ($url_arr as $url) {
If (empty ($url)) continue;
Echo "Get ". $url."..." ;
$html_str = get_url ($url);
Echo $html_str;
Echo $url;
The exit;
$obj_str = get_content_object ($html_str);
Echo "OK \ n";
Fwrite ($fp, $obj_str);
}
Fclose ($fp);
}

// traverses the directory to get the contents of the file
The function get_dir ($save_file, $dir) {
$dp = opendir ($dir);
If (file_exists ($save_file)) @ unlink ($save_file);
$fp = fopen($save_file, "a+") or die("Open save file ". $save_file." failed");
While (($file = readdir ($dp))! = false) {
If ($file! = ". "& $file! = ".." ) {
Echo "Read file ". $file."..." ;
$file_content = file_get_contents($dir. $file);
$obj_str = get_content_object ($file_content);
Echo "OK \ n";
Fwrite ($fp, $obj_str);
}
}
Fclose ($fp);
}


// gets the specified url content
The function get_url ($url) {
$reg = '/ ^ HTTP: \ \ [^ \] / / / + $/';
If (! Preg_match ($reg, $url)) die($url." invalid");
$fp = fopen ($url, "r") or die (" Open the url: ". $url. The "failed.");
While (= $fc fread ($fp, 8192)) {
$content. = $fc;
}
Fclose ($fp);
If (empty ($content)) {
Die ("Get url: ". $url." content failed.");
}
Return the $content;
}

// gets the specified web page using socket
The function get_content_by_socket ($url, $host) {
$fp = fsockopen($host, 80) or die("Open ". $url." failed");
$header = "GET /".$url." HTTP/1.1\r\n";
$header. = "Accept: * / * \ r \ n";
$header. = "Accept - Language: useful - cn \ r \ n";
$header. = "accept-encoding: gzip, deflate\r\n";
$header. = "user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon. InfoPath. 1; The.net CLR 2.0.50727) \ r \ n ";
$header. = "Host: ". $Host."\r\n";
$header. = "Connection: Keep - the Alive \ r \ n";
/ / $header. = "cookies: cnzz02 = 2; Rtime = 1; Ltime = 1148456424859; Cnzz_eid = 56601755 - \ r \ n \ r \ n ";
$header. = "Connection: Close \ r \ n \ r \ n";

Fwrite ($fp, $header);
While (! The feof ($fp)) {
$contents. = the fgets ($fp, 8192);
}
Fclose ($fp);
Return $contents;
}


// gets the url in the specified content
The function get_content_url ($host_url, $file_contents) {

/ / $reg = '/ ^ (# | javascript. *? | FTP: \ \ /. / + | HTTP: \ \ /. / + |. *? Href. *? | play. *? | index. *? |. *? Asp) + $/ I ';
/ / $reg = '/ ^ (down. *? | \. HTML \ d + _ \ d + \. HTM. *?) $/ I ';
$rex = "/ ([hH] [rR] [eE] [Ff]) = \ \ s * * * [' s \"] ([^ > \ '\' s] +) [\ "' >] * \ s * / I ";
$reg = '/ ^ (down. *? \. HTML) $/ I ';
Preg_match_all ($rex, $file_contents, $r);
$result = ""; / / array ();
The foreach ($r as $c) {
If (is_array ($c)) {
Foreach ($c as $d) {
If (preg_match ($reg, $d)) {$result. = $host_url. $d. "\ n"; }
}
}
}
Return the $result;
}

// gets the multimedia file in the specified content
The function get_content_object ($STR, $STR = "| - : * * : - |") {
$regx = "/ href = \ \ s * * * [' s \"] ([^ > \ '\' s] +) [\ "' >] * \ s * (< B> . *? < \ / b>) / I ";
Preg_match_all ($regx, $STR, $result);

If (count ($result) = = 3) {
$result [2] = str_replace (" < B> Multimedia: ", ", $result[2]);
$result [2] = str_replace (" < / b>" , "", $result [2]);
$result = $result [1] [0]. $split. $result [2] [0]. "\ n";
}
Return the $result;
}

? >

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

When there are multiple ips for the same domain name, PHP gets a function for the content of a remote web page

FGC simply reads in and encapsulates everything
Fopen also does some encapsulation, but requires you to loop through all the data.
Fsockopen this is the socket operation of the straight board.
If you're just reading an HTML page, FGC is better.
A file_get_content function will not work if the company is accessing the Internet through a firewall. Of course, it is possible to write HTTP requests directly to the proxy through some socket operations, but it is more cumbersome.
If you can confirm that the file is small, you can choose either fopen,join(",file($file)), or join(",file($file)). . For example, if you only work with files smaller than 1k, you might want to use file_get_contents instead.

If you are sure that the file is large, or if you cannot determine the size of the file, it is best to use a file stream. There is no obvious difference between fopen a 1K file and fopen a 1G file. Long content allows you to take longer to read, rather than letting the script die.

----------------------------------------------------
http://www.phpcake.cn/archives/tag/fsockopen
PHP can get remote web content in a variety of ways, such as with the built-in file_get_contents, fopen, and so on.

< ? PHP

Echo file_get_contents (" http://img.jb51.net/abc.php ");
? >
However, in load balancing such as DNS polling, the same domain name may correspond to multiple servers, multiple IP. Suppose that img.jb51.net is resolved to three IP's of 72.249.146.213, 72.249.146.214 and 72.249.146.215 by DNS. Every time the user visits img.jb51.net, the system will visit one of the servers according to the corresponding load balancing algorithm.
While working on a video project last week, I encountered a requirement to access a PHP interface on each server in turn (let's say abc.php) to query the server's transport status.

Cannot use the file_get_contents visit http://img.jb51.net/abc.php directly at this moment, because it may have been repeated access to a server.

And adopt the method of ordinal visit http://72.249.146.213/abc.php, http://72.249.146.214/abc.php, http://72.249.146.215/abc.php, in the three servers Web Server with multiple virtual host, is no good.

Setting local hosts is also not acceptable because hosts cannot set multiple ips to correspond to the same domain name.

The only way to do that is through PHP and the HTTP protocol: when accessing abc.php, add the img.jb51.net domain name to the header. So, I wrote the following PHP function:
 
<?php 

 
function HttpVisit($ip, $host, $url) 
{ 
$errstr = ''; 
$errno = ''; 
$fp = fsockopen ($ip, 80, $errno, $errstr, 90); 
if (!$fp) 
{ 
return false; 
} 
else 
{ 
$out = "GET {$url} HTTP/1.1rn"; 
$out .= "Host:{$host}rn"; 
$out .= "Connection: closernrn"; 
fputs ($fp, $out); 

while($line = fread($fp, 4096)){ 
$response .= $line; 
} 
fclose( $fp ); 

//Remove the Header information
$pos = strpos($response, "rnrn"); 
$response = substr($response, $pos + 4); 

return $response; 
} 
} 

//Call method:
$server_info1 = HttpVisit("72.249.146.213", "img.jb51.net", "/abc.php"); 
$server_info2 = HttpVisit("72.249.146.214", "img.jb51.net", "/abc.php"); 
$server_info3 = HttpVisit("72.249.146.215", "img.jb51.net", "/abc.php"); 
?> 

Related articles: