How to use Python’s requests module?

1. File upload

We know that requests can simulate submitting some data, among other things. Some websites require us to upload files, and we can also use requests to achieve this. For example, if we want to upload a file now, we can do this.

import requests f = {'f':open('a.text','rb')} r = requests.post('http://httpbin.org/post',files = f) print(r.text)

Let’s run the program and see what the effect is.

We can see that the response contains the files field, but the form field is empty, which proves that the file upload part will be identified by a separate files field.

2.Cookies

We can use cookies to maintain the login status. In the browser and in the developer tools, we can find the cookies field, and we can copy it directly.

We set the cookies in the headers, and then send the request to log in. The sample code is as follows:

3.SSL certificate verification

So what is an SSL certificate?

• #The SSL certificate is a digital certificate, which is an electronic copy, similar to the electronic copies of driver's licenses, passports and business licenses. It is signed by a trusted digital certificate authority (CA), and both the client and server must verify the validity of the certificate.

• The main function of an SSL certificate is to provide server authentication and data transmission encryption. It ensures that data is not stolen or tampered with during transmission, and it enhances security and prevents unauthorized persons from accessing the server.

• During the SSL certificate verification process, both the client and the server need to verify the validity of the certificate.

In addition, requests also have the function of certificate verification. When sending an HTTP request, it will check the SSL certificate. We can use the verify parameter to control whether to check this certificate. It is generally turned on by default.

So how do we write our code?

response = requests.get('http://www.baidu.com',verify = False)

4. Proxy settings

For some websites, content can still be obtained during testing. Once crawled frequently, the IP may be blocked, resulting in inaccessibility for a period of time. So, in order to prevent this from happening, we need to set up a proxy to solve the problem. The proxies parameter is used here.

Proxy is a built-in module of Python that can be used to create proxy objects for forwarding requests and responses on the network. In Python, you can use the requests library to send HTTP requests and use the proxies parameter to specify proxy objects.

The following is a sample code that demonstrates how to use the proxies parameter to set the proxy object:

import requests # 创建一个 requests 对象 r = requests.get('http://example.com') # 设置代理对象 r.proxies = {'http': 'http://proxy.example.com:80'} # 发送请求并获取响应 response = r.send() # 打印响应状态码和头部信息 print(response.status_code) print(response.headers)

In the above code, we first create a requests object and use the proxies parameter A proxy object is set. In this example, we forward the http request to a proxy server named proxy.example.com and set the proxy server address to 80.

We then sent an HTTP GET request using the send method, passing the proxy object as a parameter. Finally, we print the response status code and header information using the response.status_code and response.headers properties.

5. Timeout setting

When the network is not good, or the server responds too slowly, or even sometimes reports an error, in order to prevent the server from failing to respond in time, We can set a timeout setting, and the timeout parameter is used here.

response = requests.get('http://www.baidu.com',timeout= 30)

The timeout attribute of the request object is used to set the request timeout. By default, the value of the timeout attribute is 60, which means the request timeout is 60 seconds. If you need to change the timeout, you can set it to a smaller value, such as 30, which means the request timeout is 30 seconds.

If you need to return a response immediately after the request is sent, you can set the timeout attribute to 0. This will cause the timeout attribute to have a value of None, indicating that the request will time out forever.

The above is the detailed content of How to use Python’s requests module?. For more information, please follow other related articles on the PHP Chinese website!