Use PHP to download all images on the Internet

With the popularization of the Internet, pictures have become an indispensable part of people's daily lives. Whether in social media, news information or personal diaries, pictures play a very important role. Therefore, we naturally thought of how to use PHP to download all pictures on the Internet and provide more functions related to pictures. In the next article, we will focus on the specific methods and techniques of using PHP to download all images on the Internet.

1. Get image links on the Internet

To download images on the Internet, you first need to clarify the source of the image. Generally, images on web pages are through the img tag. Referenced, so we can get the image link by parsing the HTML page code. In PHP, you can use the CURL or file_get_contents function to get the HTML code of the page:

$url = 'https://www.example.com';
$html = file_get_contents($url);

After getting the HTML code, we can use regular expressions to match the image links in it. Suppose we want to get all links ending with .jpg, .jpeg, .png, .gif, we can use the following regular expression:

preg_match_all('/<img.*?src=["'](.+?.(?:jpg|jpeg|png|gif))["'].*?>/i', $html, $matches);
$links = array_unique($matches[1]);

In the above code, the preg_match_all function uses regular expressions to match HTML img tag in, then obtain all image links through $matches[1], and then use the array_unique function to remove duplicate links.

2. Download the image and save it locally

After getting the link to the image, we can use the curl or file_get_contents function to download the image to the local. In order to avoid downloading files with the same name, we can give each file a unique file name. A simple method is to generate a file name based on the current time and MD5 value:

foreach ($links as $link) {
    $extension = pathinfo(parse_url($link, PHP_URL_PATH), PATHINFO_EXTENSION);
    $filename = md5($link . time()) . '.' . $extension;
    file_put_contents($filename, file_get_contents($link));
}

In the above code, we use a foreach loop to traverse all image links, and use the parse_url function to obtain the path in the link (excluding the domain name part ), then use the pathinfo function to obtain the extension in the path, and finally use the file_get_contents function to download the image to the local, and use the file_put_contents function to save the downloaded file content as a local file. Since each file name is unique, there is no need to worry about duplicate names.

3. Handling download exceptions

During the process of downloading pictures, you may encounter some abnormal situations, such as the picture link is unavailable, the download speed is too slow, etc. In order to avoid these abnormal situations from causing adverse effects on the program, we can perform exception handling on the download process. For example, when a file download fails, you can print an error message or record a log:

foreach ($links as $link) {
    $extension = pathinfo(parse_url($link, PHP_URL_PATH), PATHINFO_EXTENSION);
    $filename = md5($link . time()) . '.' . $extension;
    $content = @file_get_contents($link);
    if ($content !== false) {
        file_put_contents($filename, $content);
    } else {
        error_log('Failed to download ' . $link);
    }
}

In the above code, we use the error mask @ to mask the error prompt of the file_get_contents function, and then use the judgment structure to check the download result , and use the error_log function to record error information.

4. Control download speed

When downloading images on a large scale, we also need to consider the issue of download speed. If the download speed is too fast, it may cause unnecessary load on the server, and may even be considered a malicious attack by the server. Therefore, we need to limit the download speed.

In order to control the download speed, we can use the sleep function to pause the program for a period of time. For example, we can set a pause of 1 second after downloading a picture to ensure that the program will not place a heavy load on the server while downloading pictures:

foreach ($links as $link) {
    $extension = pathinfo(parse_url($link, PHP_URL_PATH), PATHINFO_EXTENSION);
    $filename = md5($link . time()) . '.' . $extension;
    $content = file_get_contents($link);
    if ($content !== false) {
        file_put_contents($filename, $content);
    } else {
        error_log('Failed to download ' . $link);
    }
    sleep(1);
}

In the above code, we use the sleep function to pause the program for 1 second.

Summary

In this article, we explored in detail how to use PHP to download all images on the Internet. The specific idea is to first obtain all image links, then loop through all links, use the file_get_contents function to download the images and save them locally, and finally perform exception handling and speed control on the download process. Although this article only provides a simple PHP implementation, I believe that readers can better understand and master the implementation process of image downloading through the introduction of this article, and help readers have a better idea and solution when encountering related problems. .

The above is the detailed content of Use PHP to download all images on the Internet. For more information, please follow other related articles on the PHP Chinese website!