网页爬虫 - 在python中使用urllib2.urlopen获取的html网页不完整
阿神
阿神 2017-04-18 09:56:27
0
1
899

http://apk.hiapk.com/appinfo/...
我想要爬取这个网页中的用户评论
但是却发现使用urllib2.urlopen(request)获取的html页面不完整

代码如图

输出如图

但是实际上这个页面里面是有东西的


请问一下为什么获取到的html不全呢

阿神
阿神

闭关修行中......

reply all (1)
Ty80

It’s not that the retrieval is incomplete. The comments on this page are generated through javascript post-loading. The requested html is returned in urllib2.urlopen and the javascript in the page will not be executed

You can directly request http://apk.hiapk.com/web/api.... This address can return all comments, it is still json, it is easy to handle

    Latest Downloads
    More>
    Web Effects
    Website Source Code
    Website Materials
    Front End Template
    About us Disclaimer Sitemap
    php.cn:Public welfare online PHP training,Help PHP learners grow quickly!