Use PHP curl to simulate a browser to grab website information

Use PHP curl to simulate a browser to grab website information_PHP tutorial

WBOY

Release： 2016-07-13 10:25:43

Original

1228 people have browsed it

Official explanation
curl is a file transfer tool that uses URL syntax to work in command line mode. Curl is a file transfer tool that uses URL syntax to work in command line mode.
It supports many protocols: FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE and LDAP. curl also supports HTTPS authentication, HTTP POST method, HTTP PUT method, FTP upload, kerberos authentication, HTTP upload, proxy server, cookies, username/password authentication, download file resumable upload,
upload file resumable upload , http proxy server pipeline (proxy tunneling), it even supports IPv6, socks5 proxy server, upload files to FTP server through http proxy server, etc., the function is very powerful.

Explanation of curl one

Application of curl function in PHP
In short, there are four steps in total
curl_init();
curl_setopt();
curl_exec() ;
curl_close();

The most important command is curl_setopt();

A simple post request example
index.php

Copy code The code is as follows:

$url = "http://www.mytest.com/curl/login.php" ; //Requested url address
$user = "zkg111"; //User name
$pass = "123456";
$postdata = "user_name=".$user."&password=". $pass; //The requested data is separated by ampersand
$curl = curl_init(); //Open curl
curl_setopt($curl, CURLOPT_URL, $url); //Set the request address
curl_setopt ($curl, CURLOPT_RETURNTRANSFER, 1); //Whether to output 1 or true, not to output 0 or false to output
curl_setopt($curl, CURLOPT_POST, 1); //Whether to use the post method to request
curl_setopt($curl , CURLOPT_POSTFIELDS, $postdata); //post data

echo $data = curl_exec($curl); //Perform curl operation
curl_close($curl);
?>

Here is a simple example. I opened the Band of Brothers forum at random, and then simulated logging in to the Band of Brothers forum. If I need to post, the principle is the same, transfer the page and submit the data
Special attention should be paid to the fact that the cookie storage directory under Windows 7 must be in the ./temp directory. I created a new folder myself at first and found that the saving was correct, but the cookie was not read correctly. I still have a lot of questions about this. I asked questions in
, but no one got the right answer. After struggling for several days, it was time to save the file in the ./temp directory. I would like to remind other friends not to mess around like me

Copy code The code is as follows:

$url = "http://bbs.lampbrother.net/login.php";
$urls = "http://bbs.lampbrother.net";
$lgt = 0;
$user = "XXXX";
$pass = "XXXX";
$question = 0;
$hideid = 1;
$cookie_file = tempnam('./temp','cookie');
$postdata = "forward=&jumpurl=".$urls."&step=2&lgt =".$lgt."&pwuser=".$user."&pwpwd=".$pass."&question=".$question."&answer=&hideid=".$hideid;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$data = curl_exec($ch);
curl_close($ch);
//echo $data;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://bbs .lampbrother.net/');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_exec($ch);
curl_close($ch);
?>