phpSpider Advanced Guide: How to use regular expressions to extract web content?

phpSpider Advanced Guide: How to use regular expressions to extract web content?

Foreword:
When developing web crawlers, we often need to extract specific content from web pages. Regular expressions are a powerful tool that can help us perform pattern matching in web pages and extract the required content quickly and accurately. This article will give you an in-depth understanding of how to use regular expressions to extract web content in PHP, and comes with example code.

1. Basic syntax of regular expressions
Regular expression is a way to describe character patterns. Use regular expressions to flexibly match, find, and replace strings. The following is some basic syntax of regular expressions:

1. Character matching:
2. .: Matches any character
3. []: Matches any character within brackets
4. w: Matches any letter, number or underscore
5. d: Matches any number
6. s: Matches any blank character
7. : Matches Word boundaries
8. Repeat match:
9. • : Match 0 or more repetitions of the previous character
10. • : Matches 1 or more repetitions of the previous character
11. ?: Matches 0 or 1 repetition of the previous character
12. {n} : Matches exactly n repetitions of the previous character
13. {n,} : Matches at least n repetitions of the previous character
14. {n,m} : Matches at least n times of the previous character , repeat
15. up to m times Escape characters:
16. : Escape special characters, for example. Indicates matching dot

2. Use the preg_match function for regular matching
PHP provides a series of functions for processing regular expressions, the most commonly used of which is the preg_match function. This function is used to perform regular string matching. The following is the basic usage of the preg_match function:

$pattern = '/正则表达式/'; $string = '要匹配的字符串'; $result = preg_match($pattern, $string, $matches);

Among them, $pattern is the regular expression to be matched, $string is the string to be matched, $result is the Boolean value of the matching result, and $matches is to store the matches. Array of results.

3. Example Demonstration
Let us use an example to illustrate how to use regular expressions to extract web page content.

Suppose we want to extract all links from the following target web page:

  Link 1 Link 2 Link 3  

We can use the following regular expression to match all links:

$pattern = '/(.*?)/';

Then, we You can use the preg_match_all function to store all matching results in a two-dimensional array:

$pattern = '/(.*?)/'; $string = 'Link 1Link 2Link 3'; preg_match_all($pattern, $string, $matches); var_dump($matches[1]); // 输出所有链接

After executing this code, we will get the following output:

array(3) { [0]=> string(23) "https://www.example.com/link1" [1]=> string(23) "https://www.example.com/link2" [2]=> string(23) "https://www.example.com/link3" }

In this way, we succeeded All links are extracted from the web page.

4. Notes
It is worth noting that when using regular expressions for crawler development, you should pay attention to the following points:

1. Greedy and non-greedy
   By default, regular expression repeat matching is greedy, that is, it matches as many times as possible. We can use ? to change greedy matching to non-greedy matching.

For example, the following regular expression will greedily match the entire string "abcdef":

$pattern = '/a.*b/'; $string = 'abcdef'; preg_match($pattern, $string, $matches); var_dump($matches[0]); // 输出'abcdef'

If we change greedy matching to non-greedy matching, only The shortest substring:

$pattern = '/a.*?b/'; $string = 'abcdef'; preg_match($pattern, $string, $matches); var_dump($matches[0]); // 输出'ab'

2. Line breaks in HTML tags
   When extracting web content, you often encounter line breaks contained in HTML tags. In order to match content containing newlines, we can add the s modifier to the regular expression pattern:

$pattern = '/
(.*)
/s'; $string = '
This is a paragraph.
 
This is another paragraph.
'; preg_match_all($pattern, $string, $matches); var_dump($matches[1]); // 输出两个段落的内容

Summary:
Through the introduction of this article, you already understand how to use regular expressions Expression method to extract web page content in PHP. Regular expressions are a very powerful tool for efficiently extracting the information you need. I hope this content can help you better develop web crawlers.

The above is the detailed content of phpSpider Advanced Guide: How to use regular expressions to extract web content?. For more information, please follow other related articles on the PHP Chinese website!