Introduction to php snoopy collection class_PHP tutorial

Snoopy is a php class used to simulate some simple functions of the browser. It can obtain web content, send forms and other operations. The correct operation of Snoopy requires that your server's PHP version is 4 or above and supports PCRE (Perl Compatible Regular Expressions), which is supported by basic LAMP services. Since it is a PHP class and does not need to be extended, it is the best choice when the server does not support curl.

Snoopy’s features:

1. Fetch the content of the web page fetch

2. Fetch the text content of the web page (remove HTML tags) fetchtext

3. Fetch web page links, form fetchlinks fetchform

4. Support proxy host

5. Support basic username/password verification

6. Support setting user_agent, referer (source), cookies and header content (header file)

7. Supports browser redirection and can control redirection depth

8. Can expand links in web pages into high-quality URLs (default)

9. Submit data and get the return value

10. Support tracking HTML framework

11. Support passing cookies when redirecting

Download address of Snoopy class: http://sourceforge.net/projects/snoopy/

Snoopy class methods:

fetch($URI)

This is the method used to crawl the content of web pages. The $URI parameter is the URL address of the crawled web page. The fetched results are stored in $this->results. If you are scraping a frame, Snoopy will track each frame and store it in an array, then store it in $this->results.

fetchtext($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data and only return the text content in the web page.

fetchform($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the form content (form) in the web page.

fetchlinks($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return links in the web page. By default, relative links will be automatically completed and converted into full URLs.

submit($URI,$formvars)

This method sends a confirmation form to the link address specified by $URL. $formvars is an array that stores form parameters.

submittext($URI,$formvars)

This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the text content of the web page after login.

submitlinks($URI)

This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data and only return the link in the web page. By default, relative links will be automatically completed and converted into full URLs.

Snoopy class attributes: (default values ​​are in brackets)

$host The host to connect to
$port The port to connect to
$proxy_host The proxy host to use, if any
$proxy_port Proxy host port to use, if any
$agent User agent masquerading (Snoopy v0.1)
$referer source information, if available
$cookies cookies, if any
$rawheaders Other header information, if any
$maxredirs Maximum number of redirects, 0=not allowed (5)
$offsiteok whether or not to allow redirects off-site. (true)
$expandlinks Whether to complete links to complete addresses (true)
$user authentication username, if available
$pass authentication username, if available
$accept http accept types (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*)
$error where the error was reported, if any
$response_code Response code returned from the server
$headers Headers returned from the server
$maxlength The longest returned data length
$read_timeout read operation timeout (requires PHP 4 Beta 4+)
Set to 0 for no timeout
$timed_out If a read operation times out, this attribute returns true (requires PHP 4 Beta 4+)
$maxframes Maximum number of frames allowed to be tracked
$status The status of the captured http
$temp_dir Temporary file directory (/tmp) that the web server can write to
$curl_path cURL binary directory, if there is no cURL binary, set it to false

Snoopy usage example:

(1) Get the content of the specified url

$url='http://www.phpernote.com';
include('snoopy.php');
$snoopy=new Snoopy;
$snoopy->fetch($url);//获取所有内容 
echo $snoopy->results;//显示结果 
$snoopy->fetchtext //获取文本内容（去掉html代码） 
$snoopy->fetchlinks //获取页面所有链接 
$snoopy->fetchform //获取页面表单信息

(2) Submit form

include 'snoopy.php';
$snoopy=new Snoopy;
$formvars['username']='admin';
$formvars['pwd']='admin';
$action='http://www.phpernote.com';//表单提交地址 
$snoopy->submit($action,$formvars);//$formvars为提交的数组 
echo $snoopy->results;//获取表单提交后的 返回的结果 
$snoopy->submittext;//提交后只返回去除html的文本 
$snoopy->submitlinks;//提交后只返回链接

(3) Use Snoopy to disguise

$formvars['username']='admin';
$formvars['pwd']='admin';
$action='http://www.phpernote.com';
include 'snoopy.php';
$snoopy=new Snoopy;
$snoopy->cookies['PHPSESSID']='fc206b1918bd522cc863p36890e6notef7';//伪装sessionid 
$snoopy->agent='(compatible;MSIE 4.01;MSN 2.5;AOL 4.0;Windows 98)';//伪装浏览器 
$snoopy->referer='http://www.phpernote.com';//伪装来源页地址 http_referer 
$snoopy->rawheaders['Pragma']='no-cache';//cache 的http头信息 
$snoopy->rawheaders['X_FORWARDED_FOR']='127.0.0.1';//伪装ip 
$snoopy->submit($action,$formvars);
echo $snoopy->results;

Articles you may be interested in

• php prompt PHP Warning: date(): It is not safe to rely on the... Error solution
• php simple weight calculation method (suitable for lottery applications)
• The difference between php session and cookie
• Powerful PHP image processing class (watermark, transparency, scaling, sharpening , rotate, flip, cut, reverse color)
• Insert and Update statement construction classes for PHP novices
• php bom removal tool, php batch bom removal code
• PHP Curl batch multi-threaded open URL class
• phpMyAdmin Cannot start session without errors error solution