Buffers in Node.js, How to Use Buffers?

What is a Buffer?

A Buffer is a region of memory that represents a fixed-size, raw binary data outside the V8 JavaScript Engine.

You can think of a Buffer as an array of integers, where each integer represents a byte of data.

It is implemented by the Buffer class.

Why do we need Buffers?

Bufffers were introduced to help developers work with binary data in an ecosystem that traditionally only worked with strings instead of binary data.

Buffers are deeply linked with streams. When a stream process receives data faster than it can handle, it will put the data into a buffer.

A simple analogy for a buffer is when you are watching a video on YouTube and there are two red and white lines. When the white line is longer, it means the browser is loading data faster than it is being watched, and the browser will store that data in a buffer.

Creating a Buffer

A Buffer can be created using Buffer.from(), Buffer.alloc(), or Buffer.allocUnsafe():

const buf = Buffer.from('Hey!');
Buffer.from(array);
Buffer.from(arrayBuffer[, byteOffset[, length]]);
Buffer.from(buffer);
Buffer.from(string[, encoding]);

You can also initialize a buffer with a size. The following example creates a Buffer with a size of 1KB:

const buf = Buffer.alloc(1024);
// or
const buf = Buffer.allocUnsafe(1024);

Both alloc and allocUnsafe methods allocate a Buffer of the specified size in bytes. The Buffer created by alloc is initialized with zeroes, and the Buffer created by allocUnsafe is not initialized. This means that while allocUnsafe can be faster than alloc, the allocated memory segment can potentially contain old data.

If old data exists in memory, it can be accessed or leaked when the Buffer memory is read. This is what makes allocUnsafe truly unsafe and requires caution when using it.

Using Buffers

Accessing the content of a buffer

A Buffer is a byte array that can be accessed like an array:

const buf = Buffer.from('Hey!');
console.log(buf[0]); // 72
console.log(buf[1]); // 101
console.log(buf[2]); // 121

Those numbers are Unicode code points representing the characters at the buffer positions (H => 72, e => 101, y => 121).

You can print the whole content of a buffer using the toString() method:

console.log(buf.toString()); // Hey!  

Note: If you initialize a buffer with a fixed size, the previously allocated memory segment will contain random data, not an empty buffer!

Getting the length of a Buffer

Use the length property.

const buf = Buffer.from('Hey!');
console.log(buf.length); // 4

Iterating over the content of a Buffer

const buf = Buffer.from('Hey!');
for (const item of buf) {
  console.log(item); // 72 101 121 33
}

Modifying the content of a Buffer

You can modify the whole content of a Buffer using write():

const buf = Buffer.alloc(4);
buf.write('Hey!');
console.log(buf); // Hey!  

Or you can also modify the content using array-like syntax:

const buf = Buffer.from('Hey!');
buf[1] = 111; // o
console.log(buf.toString()); // Hoy!  

Copying Buffers

Buffers can be copied using the copy() method:

const buf = Buffer.from('Hey!');
let bufcopy = Buffer.alloc(4);
buf.copy(bufcopy);
console.log(bufcopy); // Hey!  

By default, copy() copies the entire Buffer. You can provide up to 3 parameters in order to specify the start position, end position, and length of the new Buffer:

const buf = Buffer.from('Hey!');
let bufcopy = Buffer.alloc(2); // allocate 2 bytes of memory
buf.copy(bufcopy, 0, 0, 2);
console.log(bufcopy.toString()); // 'He'

Slicing a Buffer

A slice of a Buffer still refers to it, meaning that when the Buffer changes, the slice also changes.

Use slice() to create a slice. The first parameter is the starting position and you can specify a second parameter for the end position:

const buf = Buffer.from('Hey!');
buf.slice(0).toString() // Hey!  
const slice = buf.slice(0, 2);
console.log(slice.toString()); // He
buf[1] = 111; // o
console.log(slice.toString()); // Ho

Summary

Buffer is a fixed-size binary data type. JavaScript developers typically don't work with this type of data as often as C, C++, or Go developers (or any programmers who use system programming languages), interacting with memory on a daily basis. Buffers are commonly found in Stream processing - a way of processing file read/write, transmitting data directly, or any type of information exchange from terminal devices efficiently. In the next article, we will explore what Streams are!