What is Cluster in Node.js? Using Cluster to increase the scalability of Node.js applications

Problem

Node.js only has one thread to execute JavaScript code, which means it can only process one JS command at a time. If an application has multiple JS code that takes time to process, a "bottleneck" can occur. For example, your application may handle lightweight tasks but with a large number of concurrent requests, resulting in continuous connection timeouts due to not being able to handle all the requests.

It can be said that part of the application's power lies in the speed of the CPU. Nowadays, CPUs are becoming more powerful and integrated with more cores. If your Node.js application only runs on one core of the CPU, it is a waste. If there is a way to make the application utilize all the CPU cores, it would be great. For a simple calculation, let's say the application can handle 1000 req/s, but if it can utilize 8 CPU cores, that number can go up to ~8000 req/s. Furthermore, it increases fault tolerance, so if one instance malfunctions, there are still 7 instances ready to handle requests.

To harness the power of multi-core processors and increase processing capacity and fault tolerance, Cluster is one of the optimization methods. Let's learn more about Cluster in the following article.

What is Cluster?

Cluster in Node.js can be used to run multiple instances of Node.js to distribute the workload among them. The difference between Cluster and Worker Threads is that Cluster isolates worker processes, while Worker Threads run multiple threads within a single Node.js instance.

In essence, the workers created by Cluster use child_process.fork() to communicate with the process that creates them through IPC (Inter-Process Communication).

Cluster supports two methods of workload distribution:

• The first method (default on all platforms except Windows) is a round-robin distribution, in which the main process (Primary) listens on a port, listens for new connections, and distributes them to workers in a round-robin fashion, with developers integrating "smart distribution methods" to avoid overloading worker processes.

• The second approach is for the main process to create a socket to listen and send it to the workers. The workers then directly handle those requests.

In theory, the second approach provides the best performance. However, in practice, socket distribution tends to become imbalanced due to changes in the operating system scheduler. Developers have observed that over 70% of all connections are distributed to only two processes out of eight.

The workers created are separate processes that can be killed or respawned depending on the program without affecting other workers. As long as there are some active workers, the server will continue to handle incoming requests and vice versa.

Implementing Cluster

To implement Cluster, we use the node:cluster module integrated into Node. Usually, the number of workers created is equal to the number of CPU cores. The following example uses Node.js 18 LTS and express.js:

const cluster = require('node:cluster');
const numCPUs = require('node:os').cpus().length;
const express = require('express');

if (cluster.isPrimary) {
  console.log(`Primary ${process.pid} is running`);

  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} exited`);
  });
} else {
  const app = express();
  app.use(express).listen(port);

  console.log(`Worker ${process.pid} started`);
}

Then start it with the node command.

$ node index.js
Primary 3596 is running
Worker 4324 started
Worker 4520 started
Worker 6056 started
Worker 5644 started

You will see the number of workers created is equal to the number of CPU cores through the node:os module.

For more details on how to implement, refer to Node.js Cluster.

In addition, we can also use the tool pm2 to deploy Node.js applications similarly to the Cluster module.

Conclusion

Cluster is the fastest and most convenient method to increase the processing power of your JavaScript application by leveraging multi-core CPUs. Deploying Cluster is quite simple using the node:cluster module integrated into Node.js. The workload distribution algorithm of Cluster ensures that the workload is coordinated to prevent overloading of workers. If you have a multi-core server and want to make the most of those cores, try deploying Cluster right now.