python method of crawling WeChat articles-Python Tutorial-php.cn

python method of crawling WeChat articles

高洛峰

Release： 2017-03-13 15:50:04

Original

2368 people have browsed it

This article shares with you a small program that uses python to crawl WeChat articles through the Sogou entrance. It is very simple and practical. Friends in need can refer to it

I think I set up a website to collect WeChat articles, but unfortunately I couldn’t find the entry link from WeChat. I looked through a lot of information on the Internet and found that everyone’s methods are generally similar, and they all use Sogou as the entry point. The following is a python code compiled by the author to crawl WeChat articles. If you are interested, please read it

#coding:utf-8
author = &#39;haoning&#39;
**#!/usr/bin/env python
import time
import datetime
import requests**
import json
import sys
reload(sys)
sys.setdefaultencoding( "utf-8" )
import re
import xml.etree.ElementTree as ET
import os
#OPENID = &#39;oIWsFtyel13ZMva1qltQ3pfejlwU&#39;
OPENID = &#39;oIWsFtw_-W2DaHwRz1oGWzL-wF9M&ext&#39;
XML_LIST = []
# get current time in milliseconds
current_milli_time = lambda: int(round(time.time() * 1000))
def get_json(pageIndex):

global OPENID
the_headers = {
&#39;User-Agent&#39;: &#39;Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36&#39;,
&#39;Referer&#39;: &#39;http://weixin.sogou.com/gzh?openid={0}&#39;.format(OPENID),
&#39;Host&#39;: &#39;weixin.sogou.com&#39;
}

url = &#39;http://weixin.sogou.com/gzhjs?cb=sogou.weixin.gzhcb&openid={0}&page={1}&t={2}&#39;.format(OPENID, pageIndex, current_milli_time()) #url
print(url)

response = requests.get(url, headers = the_headers)
# TO-DO; check if match the reg
response_text = response.text
print response_text
json_start = response_text.index(&#39;sogou.weixin.gzhcb(&#39;) + 19
json_end = response_text.index(&#39;)&#39;) - 2
json_str = response_text[json_start : json_end] #get json
#print(json_str)
# convert json_str to json object
json_obj = json.loads(json_str) #get json obj
# print json_obj[&#39;totalPages&#39;]
return json_obj
def add_xml(jsonObj):

global XML_LIST
xmls = jsonObj[&#39;items&#39;] #get item
#print type(xmls)
XML_LIST.extend(xmls) #用新列表扩展原来的列表
**[#www.oksousou.com][2]**
# ------------ Main ----------------
print &#39;play it :) &#39;
# get total pages
default_json_obj = get_json(1)
total_pages = 0
total_items = 0
if(default_json_obj):

# add the default xmls
add_xml(default_json_obj)
# get the rest items
total_pages = default_json_obj[&#39;totalPages&#39;]
total_items = default_json_obj[&#39;totalItems&#39;]
print total_pages
# iterate all pages
if(total_pages >= 2):
  for pageIndex in range(2, total_pages + 1):
    add_xml(get_json(pageIndex)) #extend
    print &#39;load page &#39; + str(pageIndex)
    print len(XML_LIST)

Copy after login

The above is the detailed content of python method of crawling WeChat articles. For more information, please follow other related articles on the PHP Chinese website!