Get URLs using PHP and cURL
Release： 2016-07-25 09:08:38
Original
1086 people have browsed it
http://css.dzone.com/articles/retrieving-urls-parallel-curl
- class Footo_Content_Retrieve_HTTP_CURLParallel
- {
- /**
- * Fetch a collection of URLs in parallell using cURL. The results are
- * returned as an associative array, with the URLs as the key and the
- * content of the URLs as the value.
- *
- * @param array $addresses An array of URLs to fetch.
- * @return array The content of each URL that we've been asked to fetch.
- **/
- public function retrieve($addresses)
- {
- $multiHandle = curl_multi_init();
- $handles = array();
- $results = array();
-
- foreach($addresses as $url)
- {
- $handle = curl_init($url);
- $handles[$url] = $handle;
-
- curl_setopt_array($handle, array(
- CURLOPT_HEADER => false,
- CURLOPT_RETURNTRANSFER => true,
- ));
-
- curl_multi_add_handle($multiHandle, $handle);
- }
-
- //execute the handles
- $result = CURLM_CALL_MULTI_PERFORM;
- $running = false;
-
- // set up and make any requests..
- while ($result == CURLM_CALL_MULTI_PERFORM)
- {
- $result = curl_multi_exec($multiHandle, $running);
- }
-
- // wait until data arrives on all sockets
- while($running && ($result == CURLM_OK))
- {
- if (curl_multi_select($multiHandle) > -1)
- {
- $result = CURLM_CALL_MULTI_PERFORM;
-
- // while we need to process sockets
- while ($result == CURLM_CALL_MULTI_PERFORM)
- {
- $result = curl_multi_exec($multiHandle, $running);
- }
- }
- }
-
- // clean up
- foreach($handles as $url => $handle)
- {
- $results[$url] = curl_multi_getcontent($handle);
-
- curl_multi_remove_handle($multiHandle, $handle);
- curl_close($handle);
- }
-
- curl_multi_close($multiHandle);
-
- return $results;
- }
- }
|
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Articles by Author
-
2024-10-22 09:46:29
-
2024-10-13 13:53:41
-
2024-10-12 12:15:51
-
2024-10-11 22:47:31
-
2024-10-11 19:36:51
-
2024-10-11 15:50:41
-
2024-10-11 15:07:41
-
2024-10-11 14:21:21
-
2024-10-11 12:59:11
-
2024-10-11 12:17:31
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
