网页爬虫 - 爬虫遇到 redirect, 303 POST , 无法用 Python requests 完成任务

Question

小弟最近正在研究如何爬取这家机票资料 https://m.tigerair.com/booking/search 从 Chrome dev tool 看到从client 端连续发送出了两个类似的 requests {代码...} {代码...} 第一个request 看起来像是用POST 发送...

PHPz · Answer

題主你看看這段能跑麼？我命令行裡一邊試一邊複製的，應該沒問題。

import requests
import re

s = requests.session()
s.headers = {
    'Host': 'm.tigerair.com',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0',
       'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
       'Accept-Language': 'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
       'Accept-Encoding': 'gzip, deflate, br',
       'Connection': 'keep-alive'
}

res = s.get(url = 'https://m.tigerair.com/booking/search',verify = False)
print res

post_data = {
    'adults':'1',
    'arrivalStation':'HKG',
    'children':'0',
    'currency':'SGD',
    'departureDate':'2016-02-08',
    'departureStation':'SIN',
    'infants':'0',
    'returnDate':'',
    'roundtrip':'false'
   }
res = s.post(url = 'https://m.tigerair.com/booking/search',data = post_data)
print res

res = s.get('https://m.tigerair.com/booking/select')
print res
#print res.content
print re.findall(r'data-journey-sell-key="([\s\S]+?)"',res.content)