A brief analysis of Node.js in-depth learning how to add hooks in the require function

How to add a hook to the require function of Node? The following article will show you how to add hooks in the require function. I hope it will be helpful to you!

Node.js is a JavaScript runtime environment based on the Chrome V8 engine. The early Node.js adopted the CommonJS module specification, and officially supported the ES Modules feature starting from Node v13.2.0. It was not until v15.3.0 that the ES Modules feature became stable and compatible with the NPM ecosystem.

This article will introduce the workflow of the require function in Node.js, how to let Node.js directly execute ts files and how to correctly hijack Node. js's require function to implement the hook function. Next, let’s first introduce the require function.

require function

Node.js application is composed of modules, and each file is a module. For the CommonJS module specification, we import modules through the require function. So when we use the require function to import a module, what happens inside the function? Here we use the call stack to understand the process of require:

As can be seen from the above figure, when using require to import the module , the load method of the Module object will be called to load the module. The implementation of this method is as follows:

// lib/internal/modules/cjs/loader.js
Module.prototype.load = function(filename) {
  this.filename = filename;
  this.paths = Module._nodeModulePaths(path.dirname(filename));

  const extension = findLongestRegisteredExtension(filename);

  Module._extensions[extension](this, filename);
  this.loaded = true;
  // 省略部分代码
};

Note: This article refers to Node.js The version corresponding to the source code is v16.13.1

In the above code, the two important steps are:

• Step 1: According to the file Find the extension name;
• Step 2: Find the matching loader in the Module._extensions object through the parsed extension name.

There are 3 different loaders built into Node.js for loading node, json and js files . node file loader

// lib/internal/modules/cjs/loader.js
Module._extensions['.node'] = function(module, filename) {
  return process.dlopen(module, path.toNamespacedPath(filename));
};

json file loader

// lib/internal/modules/cjs/loader.js
Module._extensions['.json'] = function(module, filename) {
 const content = fs.readFileSync(filename, 'utf8');
 try {
    module.exports = JSONParse(stripBOM(content));
 } catch (err) {
   err.message = filename + ': ' + err.message;
   throw err;
 }
};

js file loader

// lib/internal/modules/cjs/loader.js
Module._extensions['.js'] = function(module, filename) {
  // If already analyzed the source, then it will be cached.
  const cached = cjsParseCache.get(module);
  let content;
  if (cached?.source) {
    content = cached.source;
    cached.source = undefined;
  } else {
    content = fs.readFileSync(filename, 'utf8');
  }
  // 省略部分代码
  module._compile(content, filename);
};

Let’s analyze the more important js file loader. By observing the above code, we can know that the core processing flow of the js loader can also be divided into two steps:

• Step 1: Use fs.readFileSync Method to load the contents of the js file;
• Step 2: Use the module._compile method to compile the loaded js code.

So after understanding the above knowledge, what use does it have for us? In fact, after understanding the workflow of the require function, we can extend the Node.js loader. For example, enable Node.js to run ts files.

// register.js
const fs = require("fs");
const Module = require("module");
const { transformSync } = require("esbuild");

Module._extensions[".ts"] = function (module, filename) {
  const content = fs.readFileSync(filename, "utf8");
  const { code } = transformSync(content, {
    sourcefile: filename,
    sourcemap: "both",
    loader: "ts",
    format: "cjs",
  });
  module._compile(code, filename);
};

In the above code, we introduced the built-in module module, and then used the _extensions object of the module to register our custom ts loader.

In fact, the essence of the loader is a function. Inside the function, we use the transformSync API provided by the esbuild module to implement ts -> js Code conversion. After the code conversion is completed, the module._compile method will be called to compile the code.

After seeing this, I believe some friends have also thought of the corresponding loader in Webpack. If you want to learn more, you can read the detailed explanation with multiple pictures and understand the article Webpack Loader in one go.

Address: https://mp.weixin.qq.com/s/2v1uhw2j7yKsb1U5KE2qJA

The space is limited, so we will not introduce the specific compilation process. Let's take a look at how to make the custom ts loader take effect. To enable Node.js to execute ts code, we need to complete the registration of the custom ts loader before executing the ts code. Fortunately, Node.js provides us with a module preloading mechanism:

 $ node --help | grep preload
   -r, --require=... module to preload (option can be repeated)

That is, using the -r, --require command line configuration item, we can preload the specified module. After understanding the relevant knowledge, let's test the custom ts loader. First create a index.ts file and enter the following content:

// index.ts
const add = (a: number, b: number) => a + b;

console.log("add(a, b) = ", add(3, 5));

Then enter the following command on the command line:

$ node -r ./register.js index.ts

After the above command is successfully run, the console will Output the following:

add(a, b) =  8

很明显我们自定义的 ts 文件加载器生效了，这种扩展机制还是值得我们学习的。另外，需要注意的是在 load 方法中，findLongestRegisteredExtension 函数会判断文件的扩展名是否已经注册在 Module._extensions 对象中，若未注册的话，默认会返回 .js 字符串。

// lib/internal/modules/cjs/loader.js
Module.prototype.load = function(filename) {
  this.filename = filename;
  this.paths = Module._nodeModulePaths(path.dirname(filename));

  const extension = findLongestRegisteredExtension(filename);

  Module._extensions[extension](this, filename);
  this.loaded = true;
  // 省略部分代码
};

这就意味着只要文件中包含有效的 js 代码，require 函数就能正常加载它。比如下面的 a.txt 文件：

  module.exports = "hello world";

看到这里相信你已经了解 require 函数是如何加载模块及如何自定义 Node.js 文件加载器。那么，让 Node.js 支持加载 ts、png 或 css 等其它类型的文件，有更优雅、更简单的方案么？答案是有的，我们可以使用 pirates 这个第三方库。

pirates 是什么

pirates 这个库让我们可以正确地劫持 Node.js 的 require 函数。利用这个库，我们就可以很容易扩展 Node.js 加载器的功能。

pirates 的用法

你可以使用 npm 来安装 pirates：

npm install --save pirates

在成功安装 pirates 这个库之后，就可以利用该模块导出提供的 addHook 函数来添加钩子：

// register.js
const addHook = require("pirates").addHook;

const revert = addHook(
  (code, filename) => code.replace("@@foo", "console.log('foo');"),
  { exts: [".js"] }
);

需要注意的是调用 addHook 之后会返回一个 revert 函数，用于取消对 require 函数的劫持操作。下面我们来验证一下 pirates 这个库是否能正常工作，首先新建一个 index.js 文件并输入以下内容：

// index.js
console.log("@@foo")

然后在命令行输入以下命令：

$ node -r ./register.js index.js

当以上命令成功运行之后，控制台会输出以下内容：

console.log('foo');

观察以上结果可知，我们通过 addHook 函数添加的钩子生效了。是不是觉得挺神奇的，接下来我们来分析一下 pirates 的工作原理。

pirates 是如何工作的

pirates 底层是利用 Node.js 内置 module 模块提供的扩展机制来实现 Hook 功能。前面我们已经介绍过了，当使用 require 函数来加载模块时，Node.js 会根据文件的后缀名来匹配对应的加载器。 其实 pirates 的源码并不会复杂，我们来重点分析 addHook 函数的核心处理逻辑：

// src/index.js
export function addHook(hook, opts = {}) {
  let reverted = false;
  const loaders = []; // 存放新的loader
  const oldLoaders = []; // 存放旧的loader
  let exts;

  const originalJSLoader = Module._extensions['.js']; // 原始的JS Loader 

  const matcher = opts.matcher || null;
  const ignoreNodeModules = opts.ignoreNodeModules !== false;
  exts = opts.extensions || opts.exts || opts.extension || opts.ext 
    || ['.js'];
  if (!Array.isArray(exts)) {
    exts = [exts];
  }
  exts.forEach((ext) { 
    // ... 
  }
}

为了提高执行效率，addHook 函数提供了 matcher 和 ignoreNodeModules 配置项来实现文件过滤操作。在获取到 exts 扩展名列表之后，就会使用新的加载器来替换已有的加载器。

exts.forEach((ext) => {
    if (typeof ext !== 'string') {
      throw new TypeError(`Invalid Extension: ${ext}`);
    }
    // 获取已注册的loader，若未找到，则默认使用JS Loader
    const oldLoader = Module._extensions[ext] || originalJSLoader;
    oldLoaders[ext] = Module._extensions[ext];

    loaders[ext] = Module._extensions[ext] = function newLoader(
	  mod, filename) {
      let compile;
      if (!reverted) {
        if (shouldCompile(filename, exts, matcher, ignoreNodeModules)) {
          compile = mod._compile;
          mod._compile = function _compile(code) {
			// 这里需要恢复成原来的_compile函数，否则会出现死循环
            mod._compile = compile;
			// 在编译前先执行用户自定义的hook函数
            const newCode = hook(code, filename);
            if (typeof newCode !== 'string') {
              throw new Error(HOOK_RETURNED_NOTHING_ERROR_MESSAGE);
            }

            return mod._compile(newCode, filename);
          };
        }
      }

      oldLoader(mod, filename);
    };
});

观察以上代码可知，在 addHook 函数内部是通过替换 mod._compile 方法来实现钩子的功能。即在调用原始的 mod._compile 方法进行编译前，会先调用 hook(code, filename) 函数来执行用户自定义的 hook 函数，从而对代码进行处理。

好的，至此本文的主要内容都介绍完了，在实际工作中，如果你想让 Node.js 直接执行 ts 文件，可以利用 ts-node 或 esbuild-register 这两个库。其中 esbuild-register 这个库内部就是使用了 pirates 提供的 Hook 机制来实现对应的功能。

更多node相关知识，请访问：nodejs 教程！

The above is the detailed content of A brief analysis of Node.js in-depth learning how to add hooks in the require function. For more information, please follow other related articles on the PHP Chinese website!