How to obtain web page source code and convert encoding in php

In the world of the Internet, crawlers and data acquisition are very common needs. However, many times what we get is not the result we expect, and one of the reasons is encoding problems. How to correctly obtain the source code of a web page and perform encoding conversion?

There are many ways to obtain the source code of a web page in PHP, such as file_get_contents(), curl, etc. We choose file_get_contents() as an example here.

First of all, we need to determine the encoding format of the website. If we do not specify the encoding, PHP sets the character encoding to ISO-8859-1 by default. Therefore, by default, we need to convert the obtained web page source code from ISO-8859-1 to the encoding format we need. . The following is a simple example:

$url = "https://www.example.com";
$html = file_get_contents($url);
$html = mb_convert_encoding($html, "UTF-8", "ISO-8859-1");
echo $html;

Among them, $url is the website URL that needs to be obtained, and $html is the obtained web page source code. To convert $html to encoding format, the function used is mb_convert_encoding(). Among its parameters, the first is the string that needs to be converted, the second is the target encoding format that needs to be converted, and the third is the original encoding. Format. Here we convert it to UTF-8 encoding.

In actual development, we may encounter more complex encoding formats, such as GBK, BIG5, etc. In this case, we need to handle it according to the actual situation. The encoding format can be determined by searching for charset in HTML, for example:

<meta charset="gbk">

The encoding format is uncertain In this case, we can use the mb_detect_encoding() function in the PHP library for automatic identification. For example:

$url = "https://www.example.com";
$html = file_get_contents($url);
$charset = mb_detect_encoding($html, "UTF-8, GBK, BIG5, ISO-8859-1");
$html = mb_convert_encoding($html, "UTF-8", $charset);
echo $html;

Among them, $charset represents the automatically recognized encoding format, and converts it into UTF-8 format to output the result.

Of course, in actual development, we still need to consider many details, such as network connection timeout, HTTP status code judgment, special characters in text, etc. However, this article has provided you with a basic idea and method, and briefly demonstrated several Chinese encoding conversion methods. It is analyzed and supplemented here. I believe readers can operate according to their actual needs.

The above is the detailed content of How to obtain web page source code and convert encoding in php. For more information, please follow other related articles on the PHP Chinese website!