Home > Backend Development > PHP Tutorial > PHP多线程批量采摘下载美女图片的实现代码

PHP多线程批量采摘下载美女图片的实现代码

WBOY
Release: 2016-06-13 12:18:10
Original
1061 people have browsed it

PHP多线程批量采集下载美女图片的实现代码

使用curl的多线程,另外curl可以设置请求时间,遇到很慢的url资源,可以果断的放弃,这样没有阻塞,另外有多线程请求,效率应该比较高


下面是代码实现


/**      * curl 多线程      * @author http://www.lai18.com     * @param array $array 并行网址      * @param int $timeout 超时时间     * @return mix      */ public function Curl_http($array,$timeout='15'){      $res = array();      $mh = curl_multi_init();//创建多个curl语柄      foreach($array as $k=>$url){          $conn[$k]=curl_init($url);//初始化          curl_setopt($conn[$k], CURLOPT_TIMEOUT, $timeout);//设置超时时间          curl_setopt($conn[$k], CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0)');          curl_setopt($conn[$k], CURLOPT_MAXREDIRS, 7);//HTTp定向级别 ,7最高          curl_setopt($conn[$k], CURLOPT_HEADER, false);//这里不要header,加块效率          curl_setopt($conn[$k], CURLOPT_FOLLOWLOCATION, 1); // 302 redirect          curl_setopt($conn[$k], CURLOPT_RETURNTRANSFER,1);//要求结果为字符串且输出到屏幕上              curl_setopt($conn[$k], CURLOPT_HTTPGET, true);          curl_multi_add_handle ($mh,$conn[$k]);      }       //防止死循环耗死cpu 这段是根据网上的写法          do {              $mrc = curl_multi_exec($mh,$active);//当无数据,active=true          } while ($mrc == CURLM_CALL_MULTI_PERFORM);//当正在接受数据时          while ($active and $mrc == CURLM_OK) {//当无数据时或请求暂停时,active=true              if (curl_multi_select($mh) != -1) {                  do {                      $mrc = curl_multi_exec($mh, $active);                  } while ($mrc == CURLM_CALL_MULTI_PERFORM);              }          }      foreach ($array as $k => $url) {            if(!curl_errno($conn[$k])){             $data[$k]=curl_multi_getcontent($conn[$k]);//数据转换为array             $header[$k]=curl_getinfo($conn[$k]);//返回http头信息             curl_close($conn[$k]);//关闭语柄             curl_multi_remove_handle($mh  , $conn[$k]);   //释放资源             }else{             unset($k,$url);            }          }          curl_multi_close($mh);          return $data;   }//参数接收$callback = $_GET['callback'];$hrefs = $_GET['hrefs'];$urlarray = explode(',',trim($hrefs,','));$date = date('Ymd',time());//实例化$img = new HttpImg();$stime = $img->getMicrotime();//开始时间$data = $img->Curl_http($urlarray,'20');//列表数据mkdir('./img/'.$date,0777);foreach ((array)$data as $k=>$v){ preg_match_all("/(href|src)=(["|']?)([^ "'>]+.(jpg|png|PNG|JPG|gif))\2/i", $v, $matches[$k]); if(count($matches[$k][3])>0){  $dataimg = $img->Curl_http($matches[$k][3],'20');//全部图片数据二进制  $j = 0;  foreach ((array)$dataimg as $kk=>$vv){   if($vv !=''){    $rand = rand(1000,9999);    $basename = time()."_".$rand.".".jpg;//保存为jpg格式的文件    $fname = './img/'.$date."/"."$basename";    file_put_contents($fname, $vv);       $j++;    echo "创建第".$j."张图片"."$fname"."<br>";   }else{    unset($kk,$vv);   }  } }else{  unset($matches); }}$etime = $img->getMicrotime();//结束时间echo "用时".($etime-$stime)."秒";exit;
Copy after login


Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template