This content originally appeared on Level Up Coding - Medium and was authored by Nazarii Romankiv
As you already know from my previous articles, Node.js is built around the idea of small tasks being executed fast, so let’s look at how we can better utilize the Node.js nature with the help of the Promise Pool pattern.
Common Pattern
Before we talk about solutions, let’s understand the problem we usually face. Let’s say you have N records that you need to process, or for example, make an API request for each to get the data. Let’s see a typical example that handles such a case.
So what happens here is that we are doing async code “in parallel”, so it may look like this on the timeline.
As you can see, Promise.all() runs as long as the slowest promise in the batch. So your main thread is basically “doing nothing” and is waiting for the slowest request to finish.
What is Promise Pool
The idea under Promise Pool is to utilize the main thread of Node.js to the best of its potential. To achieve better utilization we need densely pack the API calls (or any other async task) so that we do not wait while the most extended call completes, rather we schedule the next call as soon as the first one finishes.
In that way we can achieve multiple things:
- Control the throughput of our service by setting the concurrency of the Promise Pool;
- Manage load on the downstream services by setting the concurrency of the Promise Pool;
- Increase the performance of our application; Reduce idle time of our CPU.
So let’s see how we can implement it and measure the results…
Implementing Promise Pool
Let’s start with the desired interface. We want to be able to give the Promise Pool a number of records and the max number of concurrent “requests” as well as the callback that can do the processing. After processing, we want to receive an array of results processing for each individual record.
Now, let’s start implementing it. We need to keep track of the number of records we are currently processing, how many we have processed in total, and other parameters.
Let’s add a fabric method as well as public functions to configure concurrency and specify processor callback.
We have established a base, now we need to implement the crux of Promise Pool. The idea is simple, but it can be hard to wrap your head around it when you see the implementation for the first time.
We iterate over each record in the array and wait for an available sit (basically a free sit in the number of concurrent sits we have), if we have a free sit we keep scheduling the following jobs unless all sits are occupied. In case all sits are occupied, we wait for at least one job to finish. For now, let’s ignore errors.
That’s it, now we have a promising pool, let’s do performance testing.
Performance Testing
We will emulate a processor as a function that takes from 150 to 1150 milliseconds to execute.
Then we will be processing an array of 1000 records in chunks of 50 records simultaneously. First, we test it with Promise.all method.
Then with our PromisePool implementation.
Now let’s compare the results. On average processing all records with Promise.all takes 22.6 seconds while PromisePool takes 13.4 seconds, which is around 40% faster!
Summary
Promise pool is a great pattern for processing multiple records simultaneously that can help you not only improve the performance of your application but enable you to control rate-limiting to downstream services.
What next?
Love Node.js? You can read my other articles about it:
Level Up Coding
Thanks for being a part of our community! Before you go:
- 👏 Clap for the story and follow the author 👉
- 📰 View more content in the Level Up Coding publication
- 💰 Free coding interview course ⇒ View Course
- 🔔 Follow us: Twitter | LinkedIn | Newsletter
🚀👉 Join the Level Up talent collective and find an amazing job
Promise Pool or How to Improve the Performance of Node.js was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Nazarii Romankiv
Nazarii Romankiv | Sciencx (2023-02-23T02:16:39+00:00) Promise Pool or How to Improve the Performance of Node.js. Retrieved from https://www.scien.cx/2023/02/23/promise-pool-or-how-to-improve-the-performance-of-node-js/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.