How Do I Configure Proxies with Python\'s Requests Module?

Proxies with Python's Requests Module

In using Python's Requests module, configuring proxy settings may seem a bit confusing. While the documentation briefly mentions that the 'proxies' variable should contain a dictionary mapping a protocol to a proxy URL, it doesn't specify the exact format of the URL.

Proxy URL Syntax

The proxy URL format, however, is:

protocol://ip:port

where "protocol" is one of "http", "https", or "ftp".

Dictionary Structure

Hence, the structure of the 'proxies' dictionary is as follows:

proxies = {
    "protocol": "scheme://ip:port",
    ...
}

You can specify different proxies for HTTP, HTTPS, and FTP protocols:

http_proxy  = "http://10.10.1.10:3128"
https_proxy = "https://10.10.1.11:1080"
ftp_proxy   = "ftp://10.10.1.10:3128"

proxies = { 
    "http"  : http_proxy, 
    "https" : https_proxy, 
    "ftp"   : ftp_proxy
}

Request with Proxies

To use proxies with Requests, pass the 'proxies' dictionary to the 'get()' method:

r = requests.get(url, headers=headers, proxies=proxies)

Environment Variables

On Linux and Windows, you can also set proxy settings via environment variables:

export HTTP_PROXY=10.10.1.10:3128
export HTTPS_PROXY=10.10.1.11:1080
export FTP_PROXY=10.10.1.10:3128

On Windows:

set http_proxy=10.10.1.10:3128
set https_proxy=10.10.1.11:1080
set ftp_proxy=10.10.1.10:3128

The above is the detailed content of How Do I Configure Proxies with Python\'s Requests Module?. For more information, please follow other related articles on the PHP Chinese website!