Two techniques to prevent blocking the Event Loop when handling CPU-intensive tasks

Issue

Complex calculations can often cause the Event Loop to become blocked, resulting in unresponsiveness of the server until the computation is complete.

Does this mean Node.js struggles with such calculations? In today's article, I will present two techniques to address this issue.

Partitioning

Also known as "chunking," this technique is based on the principle of dividing the calculation into smaller parts to ensure the interleaving of the Event Loop.

For example, consider an average calculation of numbers from 1 to n with a complexity of O(n):

for (let i = 0; i < n; i++)
  sum += i;
let avg = sum / n;
console.log('avg: ' + avg);

The complexity and CPU usage increases as n increases. However, by applying the Partitioning technique, the calculation can be divided into n asynchronous steps with a constant performance cost of O(1):

function asyncAvg(n, avgCB) {
  var sum = 0;
  function help(i, cb) {
    sum += i;
    if (i == n) {
      cb(sum);
      return;
    }

    setImmediate(help.bind(null, i+1, cb));
  }

  help(1, function(sum){
      var avg = sum/n;
      avgCB(avg);
  });
}

asyncAvg(n, function(avg){
  console.log('avg: ' + avg);
});

Partitioning utilizes setImmediate to perform each step of the loop asynchronously, ensuring that the Event Loop is not blocked since setImmediate is executed during the "Check" phase. To learn more, I recommend reading the article Understanding the Event Loop in Node.js (in Vietnamese). In simple terms, calculations are performed on each iteration of the Event Loop.

Offloading

If your task is more complex, Partitioning might not be sufficient. The Event Loop is responsible for ordering the execution of tasks, not directly executing them. To take advantage of multi-core processors, offload the tasks from the Event Loop.

There are two ways to apply Offloading:

• Developing a C++ addon.
• Creating and managing a Worker Pool instead of using the built-in Worker Pools of Node.js such as Child Process, Cluster, or Worker Threads...

To learn more about Worker Thread implementations, you can refer to the article What are Worker threads? Do you know when to use Worker threads in node.js? (in Vietnamese).

Conclusion

For simple tasks, such as iterating over elements of an arbitrarily long array, Partitioning can be a good choice. If your computational logic is more complex, Offloading is a better approach.

The Offloading technique incurs some overhead due to the communication between the serialized objects of the Event Loop and the Worker Pool. However, it can utilize the CPU's multiple cores.

However, if your server primarily performs heavy CPU computations, you should consider whether Node.js is the best fit. Node.js excels with asynchronous I/O tasks but may not be the optimal choice for CPU-intensive tasks.

References:

• https://nodejs.org/en/docs/guides/dont-block-the-event-loop