Asksearch_php.cn

Community Learn Tools Library Leisure

English

Popular searches：
PHP
MySQL
jquery
HTML
CSS

redis - When running a python crawler, the MySQL version is 5.7 and an error is always reported

When running the python crawler, the MySQL version is 5.7 and an error is always reported {code...}. What is the problem? How to solve it?

扔个三星炸死你 2017-06-12 09:19:42

html - "DNS lookup failed" appears when crawling, but there is no problem when opening the web page. What is happening?

RT, when using python to crawl, everything was normal at the beginning, but suddenly the web page got stuck. After waiting for a while, it returned to normal. However, when crawling again at this time, it will report "DNS lookup failed". I tried many websites, but the web page failed. It o...

PHP中文网 2017-06-14 10:50:53

Web crawler - How to crawl the pictures in the Blog Park blog using python?

I wrote a small piece of code to crawl the pictures in the Blog Park blog. This code is effective for some links, but some links report errors as soon as they are crawled. What is the reason? {code...} As shown in the figure, the image can be crawled correctly. If the url is changed to {code...}, an...

某草草 2017-05-18 10:45:39

Web crawler-Introduction to Python crawler

I am about to be a sophomore, I have studied Python by myself, and I know basic grammar. I want to learn crawling, but I feel that it involves a lot of knowledge. Is there anyone who has experienced it and can summarize what they know, or how to learn Python crawling?

欧阳克 2017-07-05 10:34:11

How does the PHP crawler access the dynamic crawler agent?

How does the php crawler access the crawler agent? Did you use python before? Is it the same access method as php?

Laical 2020-03-13 17:10:19

python - Using CrawlSpider in scrapy, urls cannot be matched

My crawler code is as follows. The rules are not obtained. I don’t know what the problem is? {Code...} Run error reminder: {Code...}

为情所困 2017-05-18 10:51:02

python - Questions about crawling images?

Today's crawlers generally parse images from HTML, but the problem is that some image paths are written in JS. How to climb this? I know that you can view the pictures inside the source by opening the f12 point in the browser, but how do you use a crawler to crawl out the resources in the source?

我想大声告诉你 2017-06-28 09:25:40

javascript - node crawls Weibo

I was new to node and wanted to write a crawler to crawl Sina Weibo comments, but I found that the page was dynamically generated by JS and could not be crawled using the http module, so I used phantomjs to crawl it (I heard it would be slower, and it took nearly a month to run) 15 minutes, this is ...

仅有的幸福 2017-06-30 10:00:08

python - How to check what kind of anti-crawler is used on a URL

URL: https://www.nvshens.com/g/22377/. Open the website directly with the browser and right-click on the image to download it. Then the image directly requested by my crawler has been blocked, and then I changed I have added headers and set up IP proxy, but it still doesn’t work. But if you look at ...

ringa_lee 2017-06-12 09:27:51

Web crawler - python3.4.1 request module reports error 'list' object has no attribute 'get'

I used Python to write a crawler that crawls IP addresses. Since the website is anti-crawler, I wrote an agent and used the thread pool to open 10 threads to crawl IP addresses. However, the error 'list' object has no attribute 'get' was reported directly. If you know how to solve it, please post yo...

世界只因有你 2017-06-12 09:24:16

Web crawler - how python crawler processes json content

If you can't see clearly, the website address is http://www.sse.com.cn/disclos... The red text is the content I need, but I can't extract it. Please tell me how to operate it.

学习ing 2017-06-22 11:51:33

linux - Is there any code written in Python that can be released, automatically brought online, or is a system for preparation and management?

Is there any code written in Python that can be released, automatically put online, or is a system for preparation and management? I want to make a similar website, but I don’t have any ideas. Can you recommend a related project? Let me study first...

大家讲道理 2017-05-16 13:31:07

python - csdn crawler review

https://github.com/xchaoinfo/fuck-login/blob/master/015%20CSDN/csdn.py According to the comments on lines 87-88, why does it need to be converted into this link when using a crawler to comment on csdn articles? This How did you get the link? I looked at the web page code and there is no relevant lin...

仅有的幸福 2017-05-18 10:53:42

In what file format are configuration files such as props developed in Python stored?

In what file format are configuration files such as props developed in Python stored?

phpcn_u1582 2017-05-18 10:49:39

python writes the xth column data to the xth file

The Python crawler needs to crawl a total of 65 pages of data, and the number of columns in each page of data is uncertain. Now I can capture the data of each column, but because the number of columns is uncertain, the file name written cannot be determined. The problem is how to write the x-th colu...

漂亮男人 2017-06-12 09:23:40

python - Application doubts about Alibaba crawler source code

1. Current problem: Nothing happens when I run the py file. What is the problem? 2. I am a complete novice, I have never been exposed to py, and I have only learned the basics of html+css. But because a project of the company needed to collect regional supplier information, I thought of the py crawl...

阿神 2017-06-22 11:51:27

How to import simple_HTML_DOM file in ThinkPHP5

I want to make a web crawler to obtain data. Now I have a simple_HTML_DOM.php file. How should I use it in TP5? Directly put the file content into common.php or how to use it?

NULL 2017-12-07 13:26:15

Python - The title of the web page contains a newline. How to extract it using regular expressions?

When using python as a web crawler for CSDN, when crawling the title of a web page, I always use the regular expression (?<=\<title\>).+?(?=\<) in CSDN No, I went to CSDN source code and took a look. The title was displayed in a new line, so the original regular expressio...

女神的闺蜜爱上我 2017-06-22 11:51:43

Web crawler - Python requests.get crawler set proxy IP address has not changed

The work requires crawling information on Amazon, but Amazon's anti-crawler is too powerful, and the same IP address will be blocked. Python version: 3.6, IDE: Pycharm 2017.1. I checked a lot of information on the Internet and read the manual of the requests library, but they are all the same method...

typecho 2017-06-12 09:24:11

python - scrapy crawls many more pages than actually enters items?

{Code...} Why do I visit more pages than actually enter items when I use scrapy to crawl? Is there any way to solve the problem that after crawling for a long time, there are not many pieces of data in the items?

PHP中文网 2017-05-18 10:47:40