dede collector Principle analysis of using PHP to make a simple content collector

I made a program for serializing novels a few days ago. Because I was afraid of the trouble of updating, I wrote a collector by the way to collect information from the Eight Route Chinese Network. The function is relatively simple and the rules cannot be customized, but the general idea is in it. Customize the rules. You can expand it yourself.
Using PHP as a collector mainly uses two functions: file_get_contents() and preg_match_all(). The former is used to remotely read web page content, but it can only be used in versions above PHP5. The latter is a regular function, used Extract the required content.
　Now let’s talk about the function implementation step by step.
Since we are collecting novels, we must first extract the book title, author, and genre. Other information can be extracted as needed.
　The goal here is "Returning to the Ming Dynasty to Become a Prince". First open the bibliography page, link: http://www.86zw.com/Book/3727/Index.aspx
　If you open a few more books, you will find that the title of the book The basic format is: http://www.86zw.com/Book/书号/Index.aspx, so we can make a start page and define an to enter the book number that needs to be collected. , in the future, you can receive the book number that needs to be collected through the format of $_POST['number']. After receiving the book number, the next thing to do is to construct the bibliography page: $url=http://www.86zw.com/Book/$_POST['number']/Index.aspx. Of course, this is an example, mainly for the purpose of It's easy to explain. It's best to check the legality of $_POST['number'] when actually making it.
　After constructing the URL, you can start collecting book information. Use the file_get_contents() function to open the bibliography page: $c Open the bibliography page, view the source file, and find ""Return to the Ming Dynasty to Be a Prince"". This is the title of the book to be extracted. Regular expression to extract the book title: /(.*?)/is, use the preg_match_all() function to extract the book title: preg_match_all("/(.*?)/is",$contents,$title);In this way, the content of $title[0][0] is the title we want (the usage of preg_match_all function can Go to Baidu to check, I won’t explain it in detail here). After fetching the book information, the next step is to fetch the chapter content. To fetch the chapter content, the first thing to do is to find the address of each chapter, then open the chapter remotely, use regular rules to fetch the content, and store it in the database or directly generate an html static file. . This is the address of the chapter list: http://www.86zw.com/Html/Book/18/3727/List.shtm. It can be seen that this is the same as the bibliography page, and there are rules to be found: http://www .86zw.com/Html/Book/Classification Number/Book Number/List.shtm. The book number has been obtained earlier. The key here is to find the classification number. The classification number can be found on the previous bibliography page. Extract the classification number:
　preg_match_all("/Html/Book/[0-9]{1,}/[0-9 ]{1,}/List.shtm/is",$contents,$typeid); This is not enough, you also need a cutting function:
The PHP code is as follows:
function cut($string,$start,$end){
$message = explode($start,$string);
$message = explode($end,$message[1]); return $message[0];}where $string is the content to be cut, $start is the start where, $end is the end place. Get the category number:
$start = "Html/Book/";
$end
= "List.shtm";
$typeid = cut($typeid[0][0],$start,$end);
$ typeid = explode("/",$typeid);[/php]
　In this way, $typeid[0] is the classification number we are looking for. The next step is to construct the address of the chapter list: $chapterurl = http://www.86zw.com/Html/Book/.$typeid[0]/$_POST[‘number’]/List.shtm. With this you can find the address of each chapter. The method is as follows:
$ustart = """;
$uend
= """;
//t represents the abbreviation of title
$tstart = ">";
$tend
= "<";
// Get the path, for example: 123.shtm, 2342.shtm, 233.shtm
preg_match_all("/"[0-9]{1,}.(shtm)"/is",$chapterurl,$url);
// Get the title, for example: Chapter 1 Nine Worlds of Good People
preg_match_all("//is",$file ,$title);
$count = count($url[0]);
for($i=0;$i<=$count;$i++)
{
$u = cut($url[0][ $i],$ustart,$uend);
$t = cut($title[0][$i],$tstart,$tend);
$array[$u] = $t;
}
　$array The array is all the chapter addresses. At this point, the collector is half done. The rest is to loop through each chapter address, read it, and then match the content. This is relatively simple and will not be described in detail here. Okay, let’s stop here today. This is my first time writing such a long article, so there will inevitably be problems with language organization. Please bear with me!