Home  >  Article  >  Backend Development  >  Concurrent CURL access and control the number of concurrencies through REDIS

Concurrent CURL access and control the number of concurrencies through REDIS

PHP中文网
PHP中文网Original
2017-04-01 15:05:382115browse

Concurrent CURL access and control the number of concurrency through REDIS

// 队列redis连接操作
    public function RedisConnect() {
        $queue = 
Yii::app()->queue_redis;
        $server = $queue->host;
        $port = $queue->port;
        $timeout = $queue->timeOut;


  
      $redis =  new Redis();
        $redis->connect($server, $port, 
$timeout); // timeout=300
        return $redis;
    }

/**  redis计数控制并发
     *
     * @param       connomains: url数组
     * 
@param       key: redis键名
     * @param       max: 总运行次数
     * @param     
  ip:  ip
     * @param       c:   c当前循环次数
     * @param       i:   
并发控制,允许同时运行多少条
     * @return
     */
    private function 
redisCount(&$connomains,$key,$ip,$max,$c,$i=20){
        $RedisConnect = 
$this->RedisConnect();
        $redis = $RedisConnect->get($key);
  
      echo "redis==".$redis;
        if(isset($redis)){
            if($redis >= $i){
                if(!empty($connomains)){
            
        $this->multi_curl($connomains);
                    $connomains = 
array();
                }
                sleep(2);
                $this->redisCount($connomains,$key,$ip,$max,$c,$i); 
            }else{
                $RedisConnect->incr($key);
            }
    
    }
        
    }

//并发多线程访问
    private function multi_curl($links){
        $mh = 
curl_multi_init();
        $instances = array();
        foreach($links as 
$type=>$link){
            $instances[$type] = 
curl_init($link['url']);
            curl_setopt($instances[$type],CURLOPT_RETURNTRANSFER, 1);
            curl_setopt($instances[$type],CURLOPT_TIMEOUT, 90);//原先90 2014/11/10
        
    curl_setopt($instances[$type],CURLOPT_POST, 1);
            curl_setopt($instances[$type],CURLOPT_POSTFIELDS, $link['data']);
            curl_multi_add_handle($mh,$instances[$type]);
        }
        
      
  // 
do{$n=curl_multi_exec($mh,$active);}while($active);//$active表示还有多少个连接要执行
    
    do {
            $mrc = curl_multi_exec($mh,$active);     
//curl_multi_exec运行结果0是成功,-1是有问题
        } while ($mrc == 
CURLM_CALL_MULTI_PERFORM);  //CURLM_CALL_MULTI_PERFORM常量,值-1
        while 
($active and $mrc == CURLM_OK) {       //CURLM_OK常量,值0
            if 
(curl_multi_select($mh) != -1) {      //curl_multi_select($mh) 
判断是否正在执行,未执行0,正在执行1,错误-1
                do {
                    $mrc = 
curl_multi_exec($mh, $active);    //$active表示还有多少个连接要执行
                } 
while ($mrc == CURLM_CALL_MULTI_PERFORM);
            }
        }
      
  
        foreach($links as $type=>$link){
            $res[$type] = 
curl_multi_getcontent($instances[$type]);
            curl_close($instances[$type]);
        }
        
        return 
$res;
    }

The above is the content of concurrent CURL access and control the number of concurrency through REDIS. For more related content, please pay attention to the PHP Chinese website (m.sbmmt.com)!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Previous article:php parsing processNext article:php parsing process