Home > Web Front-end > JS Tutorial > Nodejs crawls html page content

Nodejs crawls html page content

高洛峰
Release: 2017-02-04 10:58:19
Original
1942 people have browsed it

Without further ado, I will directly post the core code for node.js to capture the content of html pages.

The specific code is as follows:

var http = require("http");
var iconv = require('iconv-lite');
var option = {
hostname: "stockdata.stock.hexun.com",
path: "/gszl/s601398.shtml"
};
var req = http.request(option, function(res) {
res.on("data", function(chunk) {
console.log(iconv.decode(chunk, "gbk"));
});
}).on("error", function(e) {
console.log(e.message);
});
req.end();
Copy after login

Let’s look at the following nodejs to capture web page content

function loadPage(url) {
var http = require('http');
var pm = new Promise(function (resolve, reject) {
http.get(url, function (res) {
var html = '';
res.on('data', function (d) {
html += d.toString()
});
res.on('end', function () {
resolve(html);
});
}).on('error', function (e) {
reject(e)
});
});
return pm;
}
loadPage('http://www.baidu.com').then(function (d) {
console.log(d);
});
Copy after login

More For articles related to multiple Nodejs crawling html page content, please pay attention to the PHP Chinese website!


Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template