84669 person learning
152542 person learning
20005 person learning
5487 person learning
7821 person learning
359900 person learning
3350 person learning
180660 person learning
48569 person learning
18603 person learning
40936 person learning
1549 person learning
1183 person learning
32909 person learning
在抓取360问答的时候,前面抓取得好好的,但是每次抓到大约40条的时候,就打不开了,所有的问题都打不开了,直接全部跳转到首页!我抓包了看了下,是被302了,请问下,除了换ip,还有什么比较好的方法来突破这种限制?
我设置了时间间隔,现在也不行了
ringa_lee
sleep(10)Pause, don’t climb too fast~
Floor 1 is right, set the crawl delay
The simple solution is to use an agent.
Setting the time interval is generally not very useful. In the case of large-scale crawling, if an IP is accessed hundreds of times, the verification code will usually be redirected.
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
sleep(10)
Pause, don’t climb too fast~
Floor 1 is right, set the crawl delay
The simple solution is to use an agent.
Setting the time interval is generally not very useful. In the case of large-scale crawling, if an IP is accessed hundreds of times, the verification code will usually be redirected.