snoopy powerful PHP collection class usage example code

Download address: http://www.jb51.net/codes/33397.html

Some features of Snoopy:

1 Fetch the content of the web page fetch
2 Grab the content of the web page Text content (remove HTML tags) fetchtext
3 Fetch web page links, form fetchlinks fetchform
4 Support proxy host
5 Support basic username/password verification
6 Support setting user_agent, referer( origin), cookies and header content (header file)
7 supports browser redirection and can control the redirection depth
8 can expand links in web pages into high-quality URLs (default)
9 Submit data and get the return value
10 Support tracking HTML framework
11 Support passing cookies when redirecting
It requires PHP 4 or above because it is a PHP class and does not need to be expanded. It supports when the server does not support curl. The best choice is the

class method:

fetch($URI)
————–

This is the method used to grab the content of the web page.
The $URI parameter is the URL address of the crawled web page.
The fetched results are stored in $this->results.
If you are scraping a frame, Snoopy will track each frame and store it in an array, and then store it in $this->results.

fetchtext($URI)
——————

This method is similar to fetch(), the only difference is that this method will remove HTML tags and other irrelevant data, only Returns the text content in the web page.

fetchform($URI)
——————

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, only Returns the form content (form) in the web page.

fetchlinks($URI)
————-

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data. Only links in the web page are returned.
By default, relative links will be automatically completed and converted into full URLs.

submit($URI,$formvars)
————————-

This method sends a confirmation form to the link address specified by $URL. $formvars is an array that stores form parameters.

submittext($URI,$formvars)
————————–

This method is similar to submit(), the only difference is that this method will remove HTML tags And other irrelevant data, only the text content in the web page after login is returned.

submitlinks($URI)
————-

This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data. Only links in the web page are returned.
By default, relative links will be automatically completed and converted into full URLs.

Class attributes: (default values are in brackets)

$host The host to connect to
$port The port to connect to
$proxy_host The proxy host to use, if any
The proxy host port used by $proxy_port, if any
$agent User agent camouflage (Snoopy v0.1)
$referer source information, if any
$cookies cookies, if If any
$rawheaders Other header information, if any
$maxredirs Maximum number of redirects, 0=not allowed (5)
$offsiteok whether or not to allow redirects off-site. ( true)
$expandlinks Whether to complete all links to complete addresses (true)
$user authentication username, if available
$pass authentication username, if available
$accept http accepts types (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*)
$error Where to report the error, if any
$response_code The response code returned from the server
$headers Header information returned from the server
$maxlength The longest returned data length
$read_timeout Read operation timeout (requires PHP 4 Beta 4+)
Set to 0 for no timeout
$ timed_out If a read operation times out, this attribute returns true (requires PHP 4 Beta 4+)
$maxframes The maximum number of frames allowed to be tracked
$status The status of the captured http
$temp_dir web server Writable temporary file directory (/tmp)
$curl_path cURL binary directory, if there is no cURL binary, set it to false

The following is the demo

Copy the code The code is as follows:

include "Snoopy.class.php";
$snoopy = new Snoopy;
$snoopy->proxy_host = "http ://www.jb51.net";
$snoopy->proxy_port = "80";
$snoopy->agent = "(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98) ";
$snoopy->referer = "http://www.jb51.net";
$snoopy->cookies["SessionID"] = 238472834723489l;
$snoopy->cookies ["favoriteColor"] = "RED";
$snoopy->rawheaders["Pragma"] = "no-cache";
$snoopy->maxredirs = 2;
$snoopy-> ;offsiteok = false;
$snoopy->expandlinks = false;
$snoopy->user = "joe";
$snoopy->pass = "bloe";
if( $snoopy->fetchtext("http://www.jb51.net"))
{
echo "

".htmlspecialchars($snoopy->results)."} 
else 
echo "error fetching document: ".$snoopy->error."n";

The following are some code snippets:
1. Get the content of the specified url

Copy the code The code is as follows:

 
 $url = "http://www.jb51.net"; 
include("snoopy.php"); 
$snoopy = new Snoopy; 
$snoopy-> fetch($url); //Get all content
echo $snoopy->results; //Display results
//Optional following
$snoopy->fetchtext //Get text content (remove html code) 
$snoopy->fetchlinks //Get links
$snoopy->fetchform //Get form
?> 

2 form submission

Copy code The code is as follows:

 
$formvars["username"] = "admin"; 
$formvars["pwd"] = "admin"; 
$action = "http://www.jb51.net";//Form submission address
$snoopy->submit ($action,$formvars);//$formvars is the submitted array
echo $snoopy->results; //Get the results returned after the form is submitted
//The following are optional
$snoopy ->submittext; //Only return the text without HTML after submission
$snoopy->submitlinks;//Only return the link after submission
?> 

Now that it has been submitted Then we can do a lot of things with the form. Next, let’s disguise the IP and browser
3. Disguise

Copy code The code is as follows:

 
$formvars["username"] = "admin"; 
$formvars["pwd"] = "admin"; 
$action = "http:// www.jb51.net"; 
include "snoopy.php"; 
$snoopy = new Snoopy; 
$snoopy->cookies["PHPSESSID"] = 'fc106b1918bd522cc863f36890e6fff7'; //Disguise sessionid 
$snoopy->agent = "(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)"; //Disguise browser
$snoopy->referer = http://www.jb51. net; //Disguise source page address http_referer 
$snoopy->rawheaders["Pragma"] = "no-cache"; //Cache's http header information
$snoopy->rawheaders["X_FORWARDED_FOR" ] = "127.0.0.101"; //Disguise ip 
$snoopy->submit($action,$formvars); 
echo $snoopy->results; 
?> 

It turns out that we can disguise the session, disguise the browser, disguise the IP, haha, we can do a lot of things.
For example, if you vote with a verification code and IP address, you can vote continuously.
ps: Disguising the IP here is actually disguising the http header, so the IP obtained through REMOTE_ADDR cannot be disguised.
On the contrary, those who obtain the IP through the http header (the kind that can prevent proxying) can do it themselves to create ip.
A brief explanation of how to verify the code:
First use an ordinary browser to view the page and find the sessionid corresponding to the verification code.
Write down the sessionid and verification code values at the same time.
Next Just use snoopy to fake it.
Principle: Since it is the same sessionid, the verification code obtained is the same as the one entered for the first time.
4 Sometimes we may need to forge more things, snoopy completely thought of it for us

Copy code The code is as follows:

$snoopy->proxy_host = "http://www.jb51.net";
$snoopy->proxy_port = "8080"; //Use proxy
$ snoopy->maxredirs = 2; //Number of redirections
$snoopy->expandlinks = true; //Whether the completion link is often used during collection
// For example, the link is /images/taoav .gif can be changed to its full linkhttp://www.jb51.net/images/taoav.gif
$snoopy->maxframes = 5 //Maximum number of frames allowed
//Note that when grabbing frames, $snoopy->results returns an array
$snoopy-> error //Return error message
?>

snoopy powerful PHP collection class usage example code_PHP tutorial