Curl crawls web pages for data
phpcn_u68041
phpcn_u68041 2017-12-07 10:08:47
0
5
1217
phpcn_u68041
phpcn_u68041

reply all(3)
phpcn_u68041

Use curl to crawl the website and pay attention to https

ringa_lee

Yes, what was said on the first floor is very complete. These two methods are usually used to obtain page information, file_get_contents and curl request

NULL

You can use file_get_contents or curl code:

function getHTTPS($url) {
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
  curl_setopt($ch, CURLOPT_HEADER, false);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_REFERER, $url);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
  $result = curl_exec($ch);
  curl_close($ch);
  return $result;
}


  • reply The default installation of curl does not support the https protocol. Do I need to add this? curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false ); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false );
    phpcn_u68041 author 2017-12-07 16:44:56
  • reply I don’t know much about curl. This is a piece of code I copied from the Internet. I tested that it can crawl HTTPS web page data and sent it to you.
    NULL author 2017-12-12 10:18:17
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template