node.js - nodejs大批量下载图片自动中断的问题如何解决
黄舟
黄舟 2017-04-17 14:35:48
0
3
971

图片列表大概有1万个图片地址,
使用nodejs下载的时候,莫名其妙就会出现错误。

基本代码如下:

var downloadImage2 = function (src) {
          var iArr = src.split("/");
        var imageName = iArr[iArr.length - 1];
        //console.log("Download begin: " + imageSrc);

        var download = function(uri, filename, callback) {
            request.head(uri, function(err, res, body) {
                // console.log('content-type:', res.headers['content-type']);
                // console.log('content-length:', res.headers['content-length']);
if (uri) {request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);}
                
            });
        };

        download(src, "./images/" + imageName, function() {
            console.log('done: ' + index++);

        });
    
}

出现的错误也有点莫名其妙,有时候是:

events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: Invalid URI "undefined"
    at Request.init (/Users//dirread/node_modules/request/request.js:275:31)
    at new Request (/Users/dirread/node_modules/request/request.js:129:8)
    at request (/Users/dirread/node_modules/request/index.js:55:10)
    at Request._callback (/Users//down.js:21:11)
    at self.callback (/Users//dirread/node_modules/request/request.js:187:22)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
黄舟
黄舟

人生最曼妙的风景,竟是内心的淡定与从容!

reply all(3)
Ty80

Invalid URI "undefined"
Tolerate errors in src, and do not download the wrong address.

迷茫

Try using fs.writeFile instead.

洪涛

In fact, it is very simple to add a null check to the URI, but from my experience, the problem occurs when an error occurs when parsing the URL, resulting in some resource URLs not getting the correct address. It is recommended to investigate from this perspective.

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template