Detailed explanation of common APIs of Nodejs Buffer module-JS Tutorial-php.cn

This article will give you a detailed introduction to the common APIs of Nodejs Buffer module. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to everyone.

Detailed explanation of common APIs of Nodejs Buffer module

Module Overview

Buffer is the core module of node. Developers can use it to process binary data, such as file streams. Reading and writing, processing of network request data, etc.

Buffer has many APIs. This article only selects the more commonly used/easy-to-understand APIs to explain, including the creation, comparison, connection, copy, search, traversal, type conversion, interception, encoding conversion, etc. of Buffer instances. [Recommended learning: "nodejs Tutorial"]

Create

new Buffer(array)
Buffer. alloc(length)
Buffer.allocUnsafe(length)
Buffer.from(array)

through new Buffer(array)

// Creates a new Buffer containing the ASCII bytes of the string 'buffer'
const buf = new Buffer([0x62, 0x75, 0x66, 0x66, 0x65, 0x72]);

Copy after login

Verify:

var array = 'buffer'.split('').map(function(v){
    return '0x' + v.charCodeAt(0).toString(16)
});

console.log( array.join() );
// 输出：0x62,0x75,0x66,0x66,0x65,0x72

Copy after login

By Buffer.alloc(length)

var buf1 = Buffer.alloc(10);  // 长度为10的buffer，初始值为0x0
var buf2 = Buffer.alloc(10, 1);  // 长度为10的buffer，初始值为0x1

Copy after login

var buf3 = Buffer.allocUnsafe(10);  // 长度为10的buffer，初始值不确定

Copy after login

var buf4 = Buffer.from([1, 2, 3])  // 长度为3的buffer，初始值为 0x01, 0x02, 0x03

Copy after login

By Buffer.from()

Example 1: Buffer.from(array)

// [0x62, 0x75, 0x66, 0x66, 0x65, 0x72] 为字符串 "buffer" 
// 0x62 为16进制，转成十进制就是 98，代表的就是字母 b
var buf = Buffer.from([0x62, 0x75, 0x66, 0x66, 0x65, 0x72]);
console.log(buf.toString());

Copy after login

Example 2: Buffer.from(string[, encoding])

Create a buffer through string and convert the buffer into characters When stringing, remember to keep the encoding consistent, otherwise garbled characters will appear, as shown below.

var buf = Buffer.from('this is a tést');  // 默认采用utf8

// 输出：this is a tést
console.log(buf.toString());  // 默认编码是utf8，所以正常打印

// 输出：this is a tC)st
console.log(buf.toString('ascii'));  // 转成字符串时，编码不是utf8，所以乱码

Copy after login

The analysis of garbled characters is as follows:

var letter = 'é';
var buff = Buffer.from(letter);  // 默认编码是utf8，这里占据两个字节 
var len = buff.length;  // 2
var code = buff[0]; // 第一个字节为0xc3，即195：超出ascii的最大支持范围
var binary = code.toString(2);  // 195的二进制：10101001
var finalBinary = binary.slice(1);  // 将高位的1舍弃，变成：0101001
var finalCode = parseInt(finalBinary, 2);  // 0101001 对应的十进制：67
var finalLetter = String.fromCharCode(finalCode);  // 67对应的字符：C

// 同理 0xa9最终转成的ascii字符为)
// 所以，最终输出为 this is a tC)st

Copy after login

Example 3: Buffer.from(buffer)

Create a new Buffer instance and copy the buffer data to the new Go to the example.

var buff = Buffer.from('buffer');
var buff2 = Buffer.from(buff);

console.log(buff.toString());  // 输出：buffer
console.log(buff2.toString());  // 输出：buffer

buff2[0] = 0x61;

console.log(buff.toString());  // 输出：buffer
console.log(buff2.toString());  // 输出：auffer

Copy after login

buffer comparison

buf.equals(otherBuffer)

Determine whether the data stored in two buffer instances are the same. If so, return true, otherwise return false.

// 例子一：编码一样，内容相同
var buf1 = Buffer.from('A');
var buf2 = Buffer.from('A');

console.log( buf1.equals(buf2) );  // true

// 例子二：编码一样，内容不同
var buf3 = Buffer.from('A');
var buf4 = Buffer.from('B');

console.log( buf3.equals(buf4) );  // false

// 例子三：编码不一样，内容相同
var buf5 = Buffer.from('ABC');  // 
var buf6 = Buffer.from('414243', 'hex');

console.log(buf5.equals(buf6));    //true

//只要比较的两者内容相同,`buf.equals(otherBuffer)` 就返回true

Copy after login

buf.compare(target[, targetStart[, targetEnd[, sourceStart[, sourceEnd]]]])

also compares two buffer instances , the difference is:

You can specify the range of a specific comparison (specified through start and end)
The return value is an integer, and the standard buf, Target size relationship

Assume the return value is

0: buf and target are the same size.
1: buf is greater than target, which means buf should be ranked after target.
-1: buf is smaller than target, which means buf should be ranked before target.

Look at the example, the official example is pretty good, just post it:

const buf1 = Buffer.from('ABC');
const buf2 = Buffer.from('BCD');
const buf3 = Buffer.from('ABCD');

// Prints: 0
console.log(buf1.compare(buf1));

// Prints: -1
console.log(buf1.compare(buf2));

// Prints: -1
console.log(buf1.compare(buf3));

// Prints: 1
console.log(buf2.compare(buf1));

// Prints: 1
console.log(buf2.compare(buf3));

// Prints: [ , ,  ]
// (This result is equal to: [buf1, buf3, buf2])
console.log([buf1, buf2, buf3].sort(Buffer.compare));

Copy after login

Buffer.compare(buf1, buf2)

Similar to buf.compare(target), it is generally used for sorting. Directly post the official example:

const buf1 = Buffer.from('1234');
const buf2 = Buffer.from('0123');
const arr = [buf1, buf2];

// Prints: [ ,  ]
// (This result is equal to: [buf2, buf1])
console.log(arr.sort(Buffer.compare));

Copy after login

Let’s talk about Buffer.from([62])

Let’s study Buffer.from(array) a little here. The following is the description of the API from the official document, that is to say, each array element corresponds to 1 byte (8 bits), and the value ranges from 0 to 255.

Allocates a new Buffer using an array of octets.

The array elements are numbers

First look at it and pass it in elements for digital scene. The following are decimal, octal, and hexadecimal, which are consistent with the expected results.

var buff = Buffer.from([62])
// 
// buff[0] === parseInt('3e', 16) === 62

Copy after login

var buff = Buffer.from([062])
// 
// buff[0] === parseInt(62, 8) === parseInt(32, 16) === 50

Copy after login

var buff = Buffer.from([0x62])
// 
// buff[0] === parseInt(62, 16) === 98

Copy after login

The array element is a string

Let’s look at the scenario where the incoming element is a string. A string starting with

0 can be interpreted as 62 or 50 (octal) when parseInt('062') is used, as seen here The first explanation was adopted.
Whether the string scenario has anything to do with parseInt() has not yet been explored in depth, just a guess. TODO (find time to study)

var buff = Buffer.from(['62'])
// 
// buff[0] === parseInt('3e', 16) === parseInt('62') === 62

Copy after login

var buff = Buffer.from(['062'])
// 
// buff[0] === parseInt('3e', 16) === parseInt('062') === 62

Copy after login

var buff = Buffer.from(['0x62'])
// 
// buff[0] === parseInt('62', 16) === parseInt('0x62') === 98

Copy after login

The array element size exceeds 1 byte

Interested students can explore on their own.

var buff = Buffer.from([256])
//

Copy after login

Buffer.from('1')

At first, you will unconsciously Buffer.from('1')[0] is equal to "1". In fact, the corresponding code of "1" is 49.

var buff = Buffer.from('1')  // 
console.log(buff[0] === 1)  // false

Copy after login

By comparison, we will know that the code 1 is a control character, which means Start of Heading.

console.log( String.fromCharCode(49) )  // '1'
console.log( String.fromCharCode(1) )  // '\u0001'

Copy after login

buffer connection: Buffer.concat(list[, totalLength])

Note: Personally, I think totalLength is quite redundant. According to the official documents, it is considered from the perspective of performance improvement. However, the internal implementation only traverses the list and accumulates length to obtain totalLength. From this point of view, performance optimization is almost negligible.

var buff1 = Buffer.alloc(10);
var buff2 = Buffer.alloc(20);

var totalLength = buff1.length + buff2.length;

console.log(totalLength);  // 30

var buff3 = Buffer.concat([buff1, buff2], totalLength);

console.log(buff3.length);  // 30

Copy after login

In addition to the performance optimization mentioned above, there are two other points to note in totalLength. Assume that the cumulative length of all buffers in the list is length

totalLength > length：返回长度为totalLength的Buffer实例，超出长度的部分填充0。
totalLength < length：返回长度为totalLength的Buffer实例，后面部分舍弃。

var buff4 = Buffer.from([1, 2]);
var buff5 = Buffer.from([3, 4]);

var buff6 = Buffer.concat([buff4, buff5], 5);

console.log(buff6.length);  // 
console.log(buff6);  // 

var buff7 = Buffer.concat([buff4, buff5], 3);

console.log(buff7.length);  // 3
console.log(buff7);  //

Copy after login

拷贝：buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])

使用比较简单，如果忽略后面三个参数，那就是将buf的数据拷贝到target里去，如下所示：

var buff1 = Buffer.from([1, 2]);
var buff2 = Buffer.alloc(2);

buff1.copy(buff2);

console.log(buff2);  //

Copy after login

另外三个参数比较直观，直接看官方例子

const buf1 = Buffer.allocUnsafe(26);
const buf2 = Buffer.allocUnsafe(26).fill('!');

for (let i = 0 ; i < 26 ; i++) {
  // 97 is the decimal ASCII value for 'a'
  buf1[i] = i + 97;
}

buf1.copy(buf2, 8, 16, 20);

// Prints: !!!!!!!!qrst!!!!!!!!!!!!!
console.log(buf2.toString('ascii', 0, 25));

Copy after login

查找：buf.indexOf(value[, byteOffset][, encoding])

跟数组的查找差不多，需要注意的是，value可能是String、Buffer、Integer中的任意类型。

String：如果是字符串，那么encoding就是其对应的编码，默认是utf8。
Buffer：如果是Buffer实例，那么会将value中的完整数据，跟buf进行对比。
Integer：如果是数字，那么value会被当做无符号的8位整数，取值范围是0到255。

另外，可以通过byteOffset来指定起始查找位置。

直接上代码，官方例子妥妥的，耐心看完它基本就理解得差不多了。

const buf = Buffer.from('this is a buffer');

// Prints: 0
console.log(buf.indexOf('this'));

// Prints: 2
console.log(buf.indexOf('is'));

// Prints: 8
console.log(buf.indexOf(Buffer.from('a buffer')));

// Prints: 8
// (97 is the decimal ASCII value for 'a')
console.log(buf.indexOf(97));

// Prints: -1
console.log(buf.indexOf(Buffer.from('a buffer example')));

// Prints: 8
console.log(buf.indexOf(Buffer.from('a buffer example').slice(0, 8)));


const utf16Buffer = Buffer.from('\u039a\u0391\u03a3\u03a3\u0395', 'ucs2');

// Prints: 4
console.log(utf16Buffer.indexOf('\u03a3', 0, 'ucs2'));

// Prints: 6
console.log(utf16Buffer.indexOf('\u03a3', -4, 'ucs2'));

Copy after login

写：buf.write(string[, offset[, length]][, encoding])

将sring写入buf实例，同时返回写入的字节数。

参数如下：

string：写入的字符串。
offset：从buf的第几位开始写入，默认是0。
length：写入多少个字节，默认是 buf.length - offset。
encoding：字符串的编码，默认是utf8。

看个简单例子

var buff = Buffer.alloc(4);
buff.write('a');  // 返回 1
console.log(buff);  // 打印 

buff.write('ab');  // 返回 2
console.log(buff);  // 打印

Copy after login

填充：buf.fill(value[, offset[, end]][, encoding])

用value填充buf，常用于初始化buf。参数说明如下：

value：用来填充的内容，可以是Buffer、String或Integer。
offset：从第几位开始填充，默认是0。
end：停止填充的位置，默认是 buf.length。
encoding：如果value是String，那么为value的编码，默认是utf8。

例子：

var buff = Buffer.alloc(20).fill('a');

console.log(buff.toString());  // aaaaaaaaaaaaaaaaaaaa

Copy after login

转成字符串: buf.toString([encoding[, start[, end]]])

把buf解码成字符串，用法比较直观，看例子

var buff = Buffer.from('hello');

console.log( buff.toString() );  // hello

console.log( buff.toString('utf8', 0, 2) );  // he

Copy after login

转成JSON字符串：buf.toJSON()

var buff = Buffer.from('hello');

console.log( buff.toJSON() );  // { type: 'Buffer', data: [ 104, 101, 108, 108, 111 ] }

Copy after login

遍历：buf.values()、buf.keys()、buf.entries()

用于对buf进行for...of遍历，直接看例子。

var buff = Buffer.from('abcde');

for(const key of buff.keys()){
    console.log('key is %d', key);
}
// key is 0
// key is 1
// key is 2
// key is 3
// key is 4

for(const value of buff.values()){
    console.log('value is %d', value);
}
// value is 97
// value is 98
// value is 99
// value is 100
// value is 101

for(const pair of buff.entries()){
    console.log('buff[%d] === %d', pair[0], pair[1]);
}
// buff[0] === 97
// buff[1] === 98
// buff[2] === 99
// buff[3] === 100
// buff[4] === 101

Copy after login

截取：buf.slice([start[, end]])

用于截取buf，并返回一个新的Buffer实例。需要注意的是，这里返回的Buffer实例，指向的仍然是buf的内存地址，所以对新Buffer实例的修改，也会影响到buf。

var buff1 = Buffer.from('abcde');
console.log(buff1);  // 

var buff2 = buff1.slice();
console.log(buff2);  // 

var buff3 = buff1.slice(1, 3);
console.log(buff3);  // 

buff3[0] = 97;  // parseInt(61, 16) ==> 97
console.log(buff1);  //

Copy after login

TODO

创建、拷贝、截取、转换、查找
buffer、arraybuffer、dataview、typedarray
buffer vs 编码
Buffer.from()、Buffer.alloc()、Buffer.alocUnsafe()
Buffer vs TypedArray

文档摘要

关于buffer内存空间的动态分配

Instances of the Buffer class are similar to arrays of integers but correspond to fixed-sized, raw memory allocations outside the V8 heap. The size of the Buffer is established when it is created and cannot be resized.