Program simulates browser request and session retention-python implementation-Python Tutorial-php.cn

Program simulates browser request and session retention-python implementation

高洛峰

Release： 2016-10-18 09:20:54

Original

1267 people have browsed it

Reading the data of a page under python can be easily implemented through urllib2

import urllib2
print urllib2.urlopen(&#39;http://www.pythontab.com&#39;).read()

Copy after login

If it involves the POST request operation of the page, you need to provide header information, submitted post data and request page.

The post data needs urllib.encode(), which actually converts the dictionary into the format of "data1=value1&data2=value2".

import urllib
import urllib2
  
HEADER = {
    &#39;User-Agent&#39; : &#39;Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0&#39;,
    &#39;Referer&#39; : &#39;http://202.206.1.163/logout.do&#39;
}
  
POSTDATA = {
    &#39;data1&#39;: &#39;value1&#39;,
    &#39;data2&#39;: &#39;value2&#39;
}
  
HOSTURL = &#39;http://xxx.com&#39;
  
enpostdata = urllib.urlencode(POSTDATA)
urlrequest = urllib2.Request(hosturl,enpostdata,HEADER)
urlresponse = urllib2.urlopen(urlrequest)
  
print urlresponse.read()

Copy after login

After the request, the browser will have a session maintenance process. The session is saved in a cookie. The next page request will put the cookie in the request header. If the cookie is lost, the session will be disconnected. .

You need to set up cookie retention under python

# cookie set
# 用来保持会话
cj = cookielib.LWPCookieJar()
cookie_support = urllib2.HTTPCookieProcessor(cj)
opener = urllib2.build_opener(cookie_support, urllib2.HTTPHandler)
urllib2.install_opener(opener)

Copy after login

The following is a library file that summarizes the above knowledge points for easy use:

# filename: analogop.py
  
#!/usr/bin/python
# -*-coding:UTF-8 -*-
  
# author: 初行
# qq: 121866673
# mail: zxbd1016@163.com
# message: I need a python job
# time: 2014/10/8
  
import urllib
import urllib2
import cookielib
  
# cookie set
# 用来保持会话
cj = cookielib.LWPCookieJar()
cookie_support = urllib2.HTTPCookieProcessor(cj)
opener = urllib2.build_opener(cookie_support, urllib2.HTTPHandler)
urllib2.install_opener(opener)
  
# default header
HEADER = {
    &#39;User-Agent&#39; : &#39;Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0&#39;,
    &#39;Referer&#39; : &#39;http://202.206.1.163/logout.do&#39;
}
  
# operate method
def geturlopen(hosturl, postdata = {}, headers = HEADER):
    # encode postdata
    enpostdata = urllib.urlencode(postdata)
    # request url
    urlrequest = urllib2.Request(hosturl, enpostdata, headers)
    # open url
    urlresponse = urllib2.urlopen(urlrequest)
    # return url
    return urlresponse

Copy after login

This is a test file because the reader has not tested it Environment, you need to build it yourself or find a website to test:

#filename: test.py
from analogop import geturlopen
  
postd = {
    &#39;usernum&#39;: &#39;2011411111&#39;,
    &#39;upw&#39;: &#39;124569&#39;,
    &#39;userip&#39;: &#39;192.168.10.1&#39;,
    &#39;token&#39;: &#39;xxx&#39;
}
  
urlread = geturlopen(&#39;http://127.0.0.1:8000/login/&#39;, postd)
print urlread.read().decode(&#39;utf-8&#39;)
urlread = geturlopen(&#39;http://127.0.0.1:8000/chafen/&#39;, {})
print urlread.read().decode(&#39;utf-8&#39;)

Copy after login