Don't block the Event Loop - Part 1

Don't block the Event Loop - Part 1

Articles in series:
  1. Don't block the Event Loop - Part 1
  2. Never Block the Event Loop
Daily short news for you
  • For a long time, I have been thinking about how to increase brand presence, as well as users for the blog. After much contemplation, it seems the only way is to share on social media or hope they seek it out, until...

    Wearing this shirt means no more worries about traffic jams, the more crowded it gets, the more fun it is because hundreds of eyes are watching 🤓

    (It really works, you know 🤭)

    » Read more
  • A cycle of developing many projects is quite interesting. Summarized in 3 steps: See something complex -> Simplify it -> Add features until it becomes complex again... -> Back to a new loop.

    Why is that? Let me give you 2 examples to illustrate.

    Markdown was created with the aim of producing a plain text format that is "easy to write, easy to read, and easy to convert into something like HTML." At that time, no one had the patience to sit and write while also adding formatting for how the text displayed on the web. Yet now, people are "stuffing" or creating variations based on markdown to add so many new formats that… they can’t even remember all the syntax.

    React is also an example. Since the time of PHP, there has been a desire to create something that clearly separates the user interface from the core logic processing of applications into two distinct parts for better readability and writing. The result is that UI/UX libraries have developed very robustly, providing excellent user interaction, while the application logic resides on a separate server. The duo of Front-end and Back-end emerged from this, with the indispensable REST API waiter. Yet now, React doesn’t look much different from PHP, leading to Vue, Svelte... all converging back to a single point.

    However, the loop is not bad; on the contrary, this loop is more about evolution than "regression." Sometimes, it creates something good from something old, and people rely on that goodness to continue the loop. In other words, it’s about distilling the essence little by little 😁

    » Read more
  • Alongside the official projects, I occasionally see "side" projects aimed at optimizing or improving the language in some aspects. For example, nature-lang/nature is a project focused on enhancing Go, introducing some changes to make using Go more user-friendly.

    Looking back, it resembles JavaScript quite a bit 😆

    » Read more

If you are writing any complex applications, reading this article will help you write more high-performance and safe applications.

In this article, we will learn how Node.js handles workloads. Also, what kinds of code are processed by the Event Loop or the Worker Pool?

Summary

Node.js processes JavaScript code in the Event Loop and provides a Worker Pool to handle expensive tasks like I/O. Node.js has good scalability, sometimes better than heavyweight approaches like the Apache server. The scalability of Node.js lies in its use of a small number of threads to handle many clients. By using fewer threads, Node.js can save more time and system memory compared to using threads (memory, context). However, since Node.js only has a few threads, we need to structure our applications to utilize them optimally.

Remember, Node.js performs well when each client-related task at any given time is "small". That means the code segments at any given time are processed as fast as possible.

This applies to both callback functions and processes being executed on the Worker Pool.

Why should I avoid blocking the Event Loop and the Worker Pool?

Node.js uses a small number of threads to handle multiple connections. In Node.js, there are two types of threads: the Event Loop (also known as the main loop, main thread, event thread, etc.) and a group of n threads in the Worker Pool.

When a thread takes a long time to execute a callback, it is called "blocking". While a thread is blocked, it cannot handle requests from any other clients. This causes 2 issues:

  • Performance: If heavy operations are frequently performed on either type of thread, the server's throughput (request/s) will be affected.
  • Security: If certain inputs can cause performance bottlenecks, the server will be completely paralyzed. This is known as a DDos attack.

A quick overview of Node.js

Node.js uses an Event-Driven architecture: it has an Event Loop to coordinate and a Worker Pool for expensive I/O tasks.

Which code runs on the Event Loop?

When starting Node.js applications, it first completes the initialization phase, imports modules, and "registers" callback functions for events. The Node.js application then enters the Event Loop, responding to client requests by executing the appropriate callback functions. These callback functions are executed synchronously and may "register" asynchronous requests to continue processing after completion. The callback functions for these asynchronous requests are also executed on the Event Loop.

The Event Loop also responds to non-blocking asynchronous requests performed by its own callback functions, such as network I/O.

In summary, the Event Loop executes the JavaScript callback functions "registered" for events and handles non-blocking asynchronous requests like network I/O.

Which code runs on the Worker Pool?

The Worker Pool in Node.js is implemented in libuv.

Node.js uses the Worker Pool to handle "expensive" tasks. This includes I/O tasks that the operating system does not provide a non-blocking version for, as well as special tasks that require multiple CPUs.

Here are the API modules that use the Worker Pool:

  • I/O:
    • DNS: dns.lookup(), dns.lookupService().
    • File system: All file system APIs except fs.FSWatcher() and synchronous APIs that use libuv's thread pool.
  • CPU:
    • Crypto: crypto.pbkdf2(), crypto.scrypt(), crypto.randomBytes(), crypto.randomFill(), crypto.generateKeyPair().
    • Zlib: All zlib APIs except synchronous APIs that use libuv's thread pool.

In many Node.js applications, these APIs are the only source of tasks for the Worker Pool. Applications and modules that use C++ add-ons can submit other tasks to the Worker Pool.

How does Node.js decide which code to run next?

In summary, the Event Loop and the Worker Pool maintain queues for pending events and tasks.

In reality, the Event Loop does not actually maintain a queue. Instead, it has a set of file descriptors that it requests the operating system to monitor, using mechanisms like epoll (Linux), kqueue (OSX), event ports (Solaris), or IOCP (Windows). These file descriptors correspond to network sockets, any files it is watching, etc. When the operating system notifies that one of these file descriptors is ready, the Event Loop translates it into the appropriate event and calls the associated callbacks.

On the other hand, the Worker Pool uses an actual queue that contains tasks waiting to be processed. A Worker fetches a task from this queue and processes it until completion. When a Worker completes a task, it emits a "At least one task has completed" event to inform the Event Loop.

What does this mean for application design?

In systems that create one thread per client like Apache, each client waiting to be served is assigned a separate thread. If a thread processing a client takes a long time, the operating system interrupts it and allows another client to execute. Thus, the operating system ensures that clients requesting a small amount of work are not delayed by clients requesting more work.

Since Node.js handles multiple clients with fewer threads, if one thread blocks processing a client's request, other pending requests may not be processed until it is completed. Therefore, treating clients fairly is the responsibility of our application. This means that you should not spend too much time on any one request from a client.

This is part of the reason why Node.js can be good at scalability, but it also means we have a responsibility to ensure fairness.

Summary

In this article, we discussed how the Event Loop and the Worker Pool handle I/O tasks. It explained why we should not block the Event Loop and what the Event Loop processes, as well as what the Worker Pool handles. In the next article, we will explore what happens if we "block" the Event Loop and the code segments that can cause the Event Loop to be blocked.

Premium
Hello

Me & the desire to "play with words"

Have you tried writing? And then failed or not satisfied? At 2coffee.dev we have had a hard time with writing. Don't be discouraged, because now we have a way to help you. Click to become a member now!

Have you tried writing? And then failed or not satisfied? At 2coffee.dev we have had a hard time with writing. Don't be discouraged, because now we have a way to help you. Click to become a member now!

View all

Subscribe to receive new article notifications

or
* The summary newsletter is sent every 1-2 weeks, cancel anytime.

Comments (0)

Leave a comment...