python 2.7 如何实现http post多并发?
PHP中文网
PHP中文网 2017-04-18 10:18:50
0
2
1019

注意是python 2.7, (multiprocessing值支持到2.6,2.7下安装会报错)

用threadpool试了一下,总是报错:

单次并发的函数

data = ['a', 'b', 'c']
#data传到threadpool.makeRequests里面的第二个参数,函数会将这个列表里面的值取出来,然后根据并发数进行并发。
def repost(data):
    response = requests.post(url, data).text
    return response

结果处理函数,只是打印出来

def postresult(requestself, responseresult):
    print(responseresult)
    #其他的处理,比如对结果插入到数据库,或者写入日志等,可以在这里写

pool = threadpool.ThreadPool(5)
requests = threadpool.makeRequests(repost, data, postresult)
**#之前出问题,是因为这里requests的名称定义和requests()方法冲突了。。。然后报错又很奇怪,没引起注意,把这里的requests改个名字就OK了。各位,承让了。
当然,下面这一样,也改一下。requests的名称。
[pool.putRequest(req) for req in requests]
pool.wait()

报错如下:

Traceback (most recent call last):
  File "C:\User\AppData\Local\Programs\Python\Python35-32\lib\site-packages\threadpool.py", line 158, in run
    result = request.callable(*request.args, **request.kwds)
  File "C:\test.py", line 29, in repost
    response = requests.post(url, data1).text
AttributeError: 'list' object has no attribute 'post'

或者有没有其他实现方法?

我靠,我靠,我靠靠靠。。。。问题解决了。看上面的备注。再见。

PHP中文网
PHP中文网

认证0级讲师

reply all(2)
阿神

First, paste the entire test.py. From the error message, you should have overwritten the request variable, causing the request to become a list type.

It is recommended not to use ThreadPool to achieve multiple concurrency. Based on the characteristics of Python, even if you use multi-threading, you can only run one instruction at the same time, and the performance improvement of multi-threading is limited.

You should first make the network asynchronous, and then implement basic ioloop. You can also implement single-threaded multi-concurrency based on asyncio or tornado

巴扎黑

2.7 does not have asyncio, you can use gevent or the like, and there are dozens of 2.7 that should have multiprocessing, but yours is IO-intensive, and using multiple processes will be counterproductive. Use a third-party library to implement coroutines

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template