node.js - Use superagent to capture UTF-8 website garbled characters
天蓬老师
天蓬老师 2017-07-06 10:35:50
0
2
1295

I use superagent cheerio to grab *Dong’s products, but the returned Chinese garbled response header is as follows
It is compressed by gzip, but it stands to reason that superagent will decompress it by default

*East product address https://item.jd.com/5025518.html

I took the product title and the result is as follows

The core code is as follows:

var url = 'https://list.jd.com/list.html?cat=670,671,672' //京东电脑 var totalData = [] // 存储总数据 superagent.get(url).end(function (err, res) { if (err) { return console.error(err) } var topicUrls = []; // 页面里面的所有url var $ = cheerio.load(res.text) // 拿到页面 $('#plist .gl-item').each(function (i, e) { $e = $(e) var href = 'https:' + $e.find('.p-img >a').attr('href') // 拿到所有url topicUrls.push(href) }) var ep = new eventproxy();// //异步调用结束后,执行某些操作 ep.after('topic_html', topicUrls.length, function (topics) { //接收res.text topics = topics.map(function (topicHtml) { var $ = cheerio.load(topicHtml, {decodeEntities: false}); return ({ title: $('.sku-name').text().trim() }); }); totalData.push(topics) console.log(totalData); }) topicUrls.forEach(function (e) { superagent.get(e).end(function (err, res) { ep.emit('topic_html', res.text); }) }) })
天蓬老师
天蓬老师

欢迎选择我的课程,让我们一起见证您的进步~~

reply all (2)
phpcn_u1582

This is not garbled code. If it is garbled code, everything is messed up. It should be a problem with your encoding.

    代言

    I directly used postman to request your address, and your situation did not appear, so I judged that it was a problem with your IDE or text editor, and set the format to utf-8.

      Latest Downloads
      More>
      Web Effects
      Website Source Code
      Website Materials
      Front End Template
      About us Disclaimer Sitemap
      php.cn:Public welfare online PHP training,Help PHP learners grow quickly!