An in-depth analysis of koa's asynchronous callback processing

1. Callback pyramid and ideal solution

We all know that JavaScript is a single-threaded asynchronous non-blocking language. Asynchronous non-blocking is certainly one of its advantages, but a large number of asynchronous operations will inevitably involve a large number of callback functions. Especially when asynchronous operations are nested, a callback pyramid problem will occur, making the code very poor readability. For example, the following example:

var fs = require('fs');

fs.readFile('./file1', function(err, data) {
  console.log(data.toString());
  fs.readFile('./file2', function(err, data) {
    console.log(data.toString());
  })
})

This example reads the contents of two files successively and prints them. The reading of file2 must be performed after the reading of file1 is completed, so the operation must be performed in the callback function of file1 reading. . This is a typical callback nesting, and there are only two levels. In actual programming, we may encounter more levels of nesting. Such code writing is undoubtedly not elegant enough.

In our imagination, a more elegant way of writing should be a way of writing that looks synchronous but is actually asynchronous, similar to the following:

var data;
data = readFile('./file1');
//下面的代码是第一个readFile执行完毕之后的回调部分
console.log(data.toString());
//下面的代码是第二个readFile的回调
data = readFile('./file2');
console.log(data.toString());

This way of writing completely avoids callback hell. In fact, koa allows us to write asynchronous callback functions in this way:

var koa = require('koa');
var app = koa();
var request=require('some module');

app.use(function*() {
  var data = yield request('http://www.baidu.com');
  //以下是异步回调部分
  this.body = data.toString();
})

app.listen(3000);

So, what makes koa so magical?

2. Generator cooperates with promise to implement asynchronous callback synchronous writing.

The key point, as mentioned in the previous article, is that generator has an effect similar to "break point". When it encounters yield, it will pause, hand over control to the function after yield, and continue execution when it returns next time.

In the koa example above, not just any object can be used after yield! Must be of a specific type. In the co function, promise, thunk functions, etc. can be supported.

In today’s article, we will use promise as an example to analyze and see how to use generator and promise to achieve asynchronous synchronization.

Let’s still analyze using the first example of reading a file. First, we need to transform the file reading function and encapsulate it into a promise object:

var fs = require('fs');

var readFile = function(fileName) {
  return new Promise(function(resolve, reject) {
    fs.readFile(fileName, function(err, data) {
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    })
  })
}

//下面是readFile使用的示例
var tmp = readFile('./file1');
tmp.then(function(data) {
  console.log(data.toString());
})

Regarding the use of promises, if you are not familiar with it, you can take a look at the syntax in es6. (I will also write an article in the near future to teach you how to use es5 syntax to implement a promise object with basic functions, so stay tuned^_^)

Simply speaking, promise can realize the callback function through promise Write it in the form of .then(callback). But our goal is to cooperate with the generator to truly achieve silky-smooth synchronized writing. How to cooperate? Look at this code:

var fs = require('fs');

var readFile = function(fileName) {
  return new Promise(function(resolve, reject) {
    fs.readFile(fileName, function(err, data) {
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    })
  })
}

//将读文件的过程放在generator中
var gen = function*() {
  var data = yield readFile('./file1');
  console.log(data.toString());
  data = yield readFile('./file2');
  console.log(data.toString());
}

//手动执行generator
var g = gen();
var another = g.next();
//another.value就是返回的promise对象
another.value.then(function(data) {
  //再次调用g.next从断点处执行generator,并将data作为参数传回
  var another2 = g.next(data);
  another2.value.then(function(data) {
    g.next(data);
  })
})

In the above code, we yield readFile in the generator, and the callback statement code is written in yield The following code is completely synchronous, realizing the idea at the beginning of the article.

After yield, what we get is another.value which is a promise object. We can use the then statement to define the callback function. The content of the function is to return the read data to the generator and continue to let the generator continue to interrupt. Execute everywhere.

Basically, this is the core principle of asynchronous callback synchronization. In fact, if you are familiar with Python, you will know that there is the concept of "coroutine" in Python, which is basically implemented using generators (I would like to doubt the generator of es6 It’s just based on python~)

However, we still execute the above code manually. So just like the previous article, we also need to implement a run function to manage the generator process so that it can run automatically!

3. Let the synchronized callback function run automatically: Writing a run function

If you carefully observe the part of the previous code that manually executes the generator, you can also find a pattern that allows us to directly write a recursive function. Instead: The

var run=function(gen){
  var g;
  if(typeof gen.next==='function'){
    g=gen;
  }else{
    g=gen();
  }

  function next(data){
    var tmp=g.next(data);
    if(tmp.done){
      return ;
    }else{
      tmp.value.then(next);
    }
  }

  next();
}

function receives a generator and allows the asynchronous execution in it to be executed automatically. Using this run function, we let the previous asynchronous code execute automatically:

var fs = require('fs');

var run = function(gen) {
  var g;
  if (typeof gen.next === 'function') {
    g = gen;
  } else {
    g = gen();
  }

  function next(data) {
    var tmp = g.next(data);
    if (tmp.done) {
      return;
    } else {
      tmp.value.then(next);
    }
  }

  next();
}

var readFile = function(fileName) {
  return new Promise(function(resolve, reject) {
    fs.readFile(fileName, function(err, data) {
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    })
  })
}

//将读文件的过程放在generator中
var gen = function*() {
  var data = yield readFile('./file1');
  console.log(data.toString());
  data = yield readFile('./file2');
  console.log(data.toString());
}
//下面只需要将gen放入run当中即可自动执行
run(gen);

Execute the above code, and you can see that the terminal prints out the contents of file1 and file2 in sequence.

It should be pointed out that the run function here only supports promises for simplicity, while the actual co function also supports thunks, etc.

In this way, the two major functions of the co function are basically introduced completely. One is the process control of the onion model, and the other is the automatic execution of asynchronous synchronization code. In the next article, I will take you to integrate these two functions and write our own co function!

The code of this article can also be found on github: https://github.com/mly-zju/async-js-demo, where promise_generator.js is the sample source code of this article.