Understanding Node.js - Performance optimization
Introduction
This is the second post related to understanding architecture of Node.js. In this article we will focus on understanding how we can optimize and manage heavy CPU tasks to make our app responsive. In previous post we understood the event loop concept and now we can try to utilize that knowledge. Also, we will cover a few best practices how to make and handle promises in efficient way.
Problem
Let's say we have a simple Express application with two GET endpoints. One endpoint responds quickly with a simple Hello World! message, while the other performs a heavy CPU-bound calculation to return a result (in this case, a complex math operation that requires significant processing power). Here’s the example:
import express from "express";
const app = express();
const port = 3000;
function calculateBlocking() {
let n = 10_000_000_000;
let sum = 0;
for (let i = 1; i < n; i++) {
sum += i * 0.31232342321;
}
let avg = sum / n;
return avg;
}
app.get("/hello", (req, res) => {
res.send("Hello World!");
});
app.get("/blocking", (req, res) => {
const result = calculateBlocking(); // CPU heavy operation
res.send("Hello from blocking " + result);
});
The problem we are facing now is that is someone would call /blocking endpoint, the event loop's thread will be busy and /hello would have to wait for calculateBlocking() to be completed and the main thread to be released.
Now, we will explore how we can fix that and make our server more responsive.
Naive solution
The simplest solution that might come to mind is to wrap the blocking function within a Promise and make it async. Let's try it and expose the operation with one more endpoint:
function calculateNonBlocking(): Promise<number> {
return new Promise((resolve, reject) => {
const result = calculateBlocking();
resolve(result);
});
}
app.get("/non-blocking-naive", async (req, res) => {
const result = await calculateNonBlocking();
res.send("Hello from non-blocking " + result);
});
Start the application and open /non-blocking-naive in one tab and /hello in a second tab.
As you might have expected (or maybe not), this approach doesn’t work. Simply making the code async by wrapping it in a Promise doesn’t change much. The code still runs synchronously, blocking the event loop and making the application unresponsive. The first takeaway here is that wrapping code in a Promise doesn’t automatically make it run asynchronously.
So, what we can do to make it run as we would expect? Actually, there are two approaches I would like to propose.
Partitioning
Partitioning is a technique that breaks down large tasks into smaller ones, which are then executed in consecutive event loop cycles, one by one, collecting the results once all are completed.
In our current blocking function, we have a long-running loop that executes in a single cycle:
let n = 10_000_000_000;
let sum = 0;
for (let i = 1; i < n; i++) {
sum += i * 0.31232342321;
}
To improve this, we can split the work to run across multiple cycles, calculating the value for the next 50,000 elements in each cycle:
let n = 10_000_000_000;
let sum = 0;
for (let i = 1; i < 50000; i++) {
sum += i * 0.31232342321;
}
for (let i = 50000; i < 100000; i++) {
sum += i * 0.31232342321;
}
//...
This approach can release the event loop after processing each segment, allowing it to handle incoming requests in between. To achieve this, we can use one of the timers functions available in Node.js. Specifically, we can utilize setImmediate to ensure that requests are handled after each poll phase. To implement this, we need to introduce a helper function that performs calculations for each segment and schedules the work for the next cycle.
function calculateNonBlockingWithImmediate(): Promise<number> {
return new Promise((resolve, reject) => {
let n = 10_000_000_000;
const partition = 50_000;
let sum = 0;
function calculateSegment(i: number, callback: (result: number) => void) {
for (let k = i; k < i + partition && k <= n; k++) {
sum += k * 0.31232342321;
}
if (i >= n) {
callback(sum);
return;
}
setImmediate(() => calculateSegment(i + partition, callback));
}
calculateSegment(1, (sum) => {
let avg = sum / n;
resolve(avg);
});
});
}
app.get("/non-blocking-immediate", async (req, res) => {
const result = await calculateNonBlockingWithImmediate();
res.send("Hello from blocking " + result);
});
By running this code, you can access both endpoints simultaneously. Applying task partitioning has made our application more responsive. This is one of the techniques you should use if your application performs heavy CPU operations, as it helps avoid blocking the event loop and ensures that your application remains responsive.
Off-loading
The other approach is called off-loading and the main purpose is to delegate heavy and long running task to a separate thread.
In Node.js, this can be done with worker threads module.
A Worker Thread is an independent JavaScript execution thread with its own event loop. Each worker runs in isolation. It means it has its own memory, global objects, and runtime without access to main program's variables. However, workers can communicate with each other and with the main thread using message passing.
Workerclass is used to create a new worker. As an argument, you need to pass on a path to worker script, e.g.new Worker('./my-worker.js).Messages- to exchange a communication between main thread and a worker you can use:on('message', callback)- to handle message sentpostMessage(data)- to post a message (either worker or main thread)
parentPort- to reference main program worker
So, let's start with creating a new file, e.g. worker.ts and adding a worker content to it
import { parentPort } from "worker_threads";
parentPort.on("message", (task) => { // receive a request to start working
console.log("started task: ", task);
const result = calculateBlocking();
console.log("completed task: ", task);
parentPort.postMessage(result); // send the results back to main thread using `parentPort`
});
function calculateBlocking() {
let n = 10_000_000_000;
let sum = 0;
for (let i = 1; i < n; i++) {
sum += i * 0.31232342321;
}
let avg = sum / n;
return avg;
}
and use the worker in our express application
import express from "express";
import { Worker } from "worker_threads";
const app = express();
const port = 3000;
const worker = new Worker("./dist/worker.js");
function calculateWithWorker(): Promise<number> {
return new Promise((resolve, reject) => {
worker.postMessage("calculate using worker");
worker.on("message", (result) => {
resolve(result);
});
worker.on("error", (error) => {
reject(error);
});
});
}
app.get("/hello", (req, res) => {
res.send("Hello World!");
});
app.get("/non-blocking-worker", async (req, res) => {
const result = await calculateWithWorker();
res.send("Hello from blocking " + result);
});
app.listen(port, () => {
console.log(`Example app listening on port ${port}`);
});
When you run the application, the results will match those in our previous example. The application will still respond to other threads, and the main event loop won’t be blocked.
The main difference between partitioning and offloading lies in where the task is executed. In the partitioning approach, the calculation remains in the same thread as the event loop. This means that if there are many tasks or if parallel processing is required, the single thread may become a bottleneck, potentially impacting application performance as it takes on more work.
In contrast, offloading involves delegating each task to a separate worker, which can also affect system resources. If we anticipate a high volume of heavy tasks, creating a pool of reusable workers is advisable to better manage resource usage and maintain performance.
Promises
Before we move to the summary, here are a few tips for handling promises that can improve your code’s performance and robustness.
Use async/await for readability
async/await syntax makes asynchronous code more readable and looks like synchronous code. It should be used in favour of then promise chaining.
async function fetchData() {
try {
const data = await fetchDataFromDatabase();
console.log(data);
} catch (error) {
console.error(error);
}
}
Avoid nesting promises whenever possible
Nesting promises can make the code unreadable, so instead of proper chain them or use async/await syntax:
fetchData()
.then(data => fetchAdditionalData(data)
.then(moreData => processResults(moreData)))
.catch(error => console.error(error));
// Instead, do this
fetchData()
.then(fetchAdditionalData)
.then(processResults)
.catch(error => console.error(error));
// or
try {
const data = await fetchData()
const moreDate = await fetchAdditionalData(data)
const result = await processResults(moreData)
} catch (error) {
console.error(error)
}
Avoid unnecessary waits
If the data can be fetch in parallel, do not use unnecessary await and process them in parallel using Promise.all/Promise.allSettled utility functions.
const posts = await fetchUserPosts(userId)
const orders = await fetchUserOrders(userId)
// Instead, do this
const [posts, orders] = await Promise.all([
fetchUserPosts(userId),
fetchUserOrders(userId)
])
Keep in mind that Promise.all fails if any promise rejects. Consider using Promise.allSettled if you want all promises to complete, regardless of success or failure.
Wrap with try/catch/finally
Make sure that your async/await code is wrapped into try/catch block in order to handle promise rejection correctly. If there are resources to be cleaned up (regardless if succeeded or failed), put that in to finally block.
Keep in mind that unhandled promise rejections can disrupt the event loop and potentially terminate the process. It is also a good practice to add global unhandled promise rejection listener to caught all promises that do not have proper error handling to report and fix them.
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled promise rejection at:', promise, 'reason:', reason);
});
Wrap callbacks into Promises
To improve code readability, wrap all callbacks in promises to avoid callback hell and handle them more efficiently.
function withCallback(param: string, callback: (result: number) => void) {
// some logic here
const result = // ... do some calculation here
callback(result);
}
function wrapWithPromise(param: string): Promise<number> {
return new Promise((resolve) => {
withCallback(param, (result) => resolve(result));
});
}
// so instead of
withCallback('example parameter', (result: number) => {
console.log('The calculated value is: ', result)
})
// it can be
const result = await wrapWithPromise('example parameter')
console.log('The calculated value is: ', result)
Summary
I hope you found this post helpful and gained a basic understanding of how to handle CPU-intensive tasks while keeping your application robust and responsive. We explored two approaches to prevent blocking the event loop during computations. Based on your requirements, you should now have a better idea of which approach to choose in different scenarios.
Also, keep in mind that simply wrapping code in a Promise does not make a function asynchronous. This is especially important when adding dependencies or third-party libraries that, theoretically, are supposed to be asynchronous based on their documentation.
In the next post, we’ll explore Node.js’s built-in profiling tools to identify bottlenecks and optimize your application. Stay tuned!