Introduction to how Python3 uses the requests package to capture and save web page source code

The example in this article describes how Python3 uses the requests package to grab and save the source code of a web page. Share it with everyone for your reference, the details are as follows:

Use the requests module of Python 3 to capture the source code of the web page and save it to a file. Example:

import requests
html = requests.get("http://www.baidu.com")
with open('test.txt','w',encoding='utf-8') as f:
 f.write(html.text)

This is a basic file saving operation, but there are several issues worth noting here:

1. Install the requests package. Enter pip install requests on the command line to install it automatically. Many people recommend using requests. The built-in urllib.request can also capture web page source code

2. Set the encoding parameter of the open method to utf-8, otherwise the saved file will be garbled.

3. If you directly output the captured content in cmd, various encoding errors will be prompted, so save it to a file for viewing.

4. The with open method is a better way of writing, which can automatically release resources after the operation is completed.

Another example:

import requests
ff = open('testt.txt','w',encoding='utf-8')
with open('test.txt',encoding="utf-8") as f:
 for line in f:
 ff.write(line)
ff.close()

This is a demonstration of reading a txt file, one line at a time, and saving it to another txt Examples in the file.

Because when printing the data read one line at a time in the command line, encoding errors will occur in Chinese, so read one line at a time and save it to another file to test whether the reading is normal. (Note that the encoding method is specified when opening)

For more introduction on how Python3 uses the requests package to capture and save the source code of web pages, please pay attention to the PHP Chinese website for related articles!