In-depth understanding of stream and pipe in nodejs

This article mainly introduces the principles and implementation of understanding the stream and pipe mechanism of nodejs. It has certain reference value. Interested friends can refer to it

Preface

Someone asked me about pipe a few days ago. I found that although I have used nodejs for a long time, I often avoid using stream because I don’t use it much each time. As a result, I have never been familiar with it. Now I am re-learning and sorting it out. related information.

Learn stream through nodeschool

nodeschool has a stream-adventure tutorial to teach the use of stream, it is very simple

Simple stream for pipe

First of all, we can locate the input to the output through the pipeline. The input and output can be the console or file stream or http request, such as

##

process.stdin.pipe(process.stdout)
process.stdin.pipe(fs.createWriteStream(path))
fs.createReadStream(path).pipe(process.stdin)

Processing in the middle of the pipe

If we want to process in the middle of the pipeline, for example, if we want to change the input string to uppercase and write it to the output, we can use some intermediate processing Frameworks, such as through2, are very convenient

var through2 = require('through2');
var stream = through2(write,end)
process.stdin
  .pipe(stream)
  .pipe(process.stdout);

function write(line,_,next){
  this.push(line.toString().toUpperCase())
  next();
})
function end(done){
  done();
})

stream is converted into a normal callback

When our input is a stream and the output is a normal function, we need to convert the input stream into an ordinary buffer. At this time, we can try the concat-stream library

var concat = require('concat-stream');

var reverseStream=concat(function(text){
  console.log(text.toString().split("").reverse().join(""));
})

process.stdin.pipe(reverseStream)

Stream in http server

Similar to stdin and fs, http is also suitable for using streams due to its characteristics, so it has similar characteristics

var http = require('http');
var server = http.createServer(function(req,res){
  req.pipe(res);
})

serves as both input and output Stream

The request framework implements the following functions. Pipe a stream into the request request, then send the content of the stream to the server, and then return it as a stream for other code to use. The implementation is as follows

var request = require('request');
var r = request.post('http://localhost:8099');
process.stdin.pipe(r).pipe(process.stdout)

Branch pipeline

The following is an example. This example puts the html elements containing loud class in the input pipeline into another pipeline for capitalization. Operation, and then finally merged into output

var trumpet = require('trumpet');
var through2 = require('through2');
var fs = require('fs');
var tr = trumpet();
var stream = tr.select('.loud').createStream();
var upper = through2(function(buf,_,next){
  this.push(buf.toString().toUpperCase());
  next();
})
stream.pipe(upper).pipe(stream);
process.stdin.pipe(tr).pipe(process.stdout);

Combined input and output stream example

The merged input and output can be the same as the previous request Use, the following example implements the use of streams to call subprocesses

var spawn = require('child_process').spawn;
var duplexer2 = require('duplexer2');

module.exports = function(cmd, args){
  var c = spawn(cmd,args)
  return duplexer2(c.stdin,c.stdout)
}

Summary

Through the above example, you can Know how to merge streams and other more complex application methods. In short, it conforms to the following characteristics as a whole:

• Stream is divided into readable and writeble

##Stream controls the flow direction through the pipe method
httpServer and httpClient and file system and process.stdin\out\err can usually be used as stream
Stream can be converted into ordinary variables by on(event), ordinary variables Can be converted into stream by write
Stream itself can be split, merged, and filtered

The above is the detailed content of In-depth understanding of stream and pipe in nodejs. For more information, please follow other related articles on the PHP Chinese website!