Use PHP to create a simple content collector code_PHP tutorial

Collector, usually called a thief program, is mainly used to capture the content of other people's web pages. Regarding the production of a collector, it is actually not difficult. You just need to remotely open the web page to be collected, and then use regular expressions to match the required content. As long as you have a little knowledge of regular expressions, you can make your own collector. .

A few days ago, I made a program for serializing novels. Because I was afraid of the trouble of updating, I wrote a collector by the way to collect information from the Eight Route Chinese Network. The function is relatively simple and the rules cannot be customized, but the general idea is the same. Inside, custom rules can be expanded by yourself.

Using PHP as a collector mainly uses two functions: file_get_contents() and preg_match_all(). The former is for remotely reading web page content, but it can only be used in versions above PHP5. The latter is Regular functions are used to extract the required content.

Now let’s talk about the function implementation step by step.

Since we are collecting novels, we must first extract the book title, author, and genre. Other information can be extracted as needed.

The goal here is "Return to the Ming Dynasty to Become a Prince", first open the bibliography page, link: http://www.86zw.com/Book/3727/Index.aspx

More If you open a few books, you will find that the basic format of the book title is: http://www.86zw.com/Book/ISBN/Index.aspx, so we can make a start page and define an , used to enter the book number that needs to be collected. In the future, the book number that needs to be collected can be received in the format of $_POST['number']. After receiving the book number, the next thing to do is to construct the bibliography page: $url=http://www.86zw.com/Book/$_POST['number']/Index.aspx. Of course, this is an example, mainly for It is easy to explain. It is best to check the legality of $_POST['number'] when actually making it.

After constructing the URL, you can start collecting book information. Use the file_get_contents() function to open the bibliographic page: $content=file_get_contents($url), so that all the contents of the bibliographic page can be read out. The next step is to match information such as book title, author, and genre. Here we take the title of the book as an example, everything else is the same. Open the bibliography page, view the source file, and find ""Back to the Ming Dynasty as a Prince"". This is the title of the book to be extracted. Regular expression to extract the book title: /(.*?)/is, use the preg_match_all() function to extract the book title: preg_match_all("/(.*?)/is",$contents,$title); In this way, the content of $title[0][0] is the title we want (the usage of preg_match_all function can Go to Baidu to check, I won’t explain it in detail here). After fetching the book information, the next step is to fetch the chapter content. To fetch the chapter content, the first thing to do is to find the address of each chapter, then open the chapter remotely, use regular rules to fetch the content, and store it in the database or directly generate an html static file. . This is the address of the chapter list: http://www.86zw.com/Html/Book/18/3727/List.shtm. It can be seen that this is the same as the bibliography page, and there are rules to be found: http://www .86zw.com/Html/Book/Classification Number/Book Number/List.shtm. The book number has been obtained previously. The key here is to find the classification number. The classification number can be found on the previous bibliographic page. Extract the classification number:

　preg_match_all("/Html/Book/[0-9]{1,} /[0-9]{1,}/List.shtm/is",$contents,$typeid); This is not enough, you also need a cutting function:
The PHP code is as follows: