When we started to work on version 2 of Kuzzle a year ago, the biggest question was what kind of promises are we going to use in Kuzzle core.
Today we have updated the 2020 benchmarks with the performances of the latest version of the v8-based JavaScript runtime: Node.js 14.
TL;DR
In the latest versions of Node.js (12 and 14):
- Use async/await for sequential execution of promises
- Use Bluebird for parallel execution of promises
In general:
- Bluebird retains the smallest memory footprint in all cases
- Promise performances are much better since Node.js 10
Doxbee benchmark
For all of our tests, we used the Doxbee benchmarks developed by the Bluebird team based on an article analyzing the different asynchronous patterns in Javascript.
These benchmarks simulate a situation in which queries are executed sequentially or in parallel with read/write blocking actions.
For each node version, we will run the sequential and then parallel benchmarks with the following commands:
$ echo ./doxbee-sequential/promises-native-async-await.js ./doxbee-sequential/promises-ecmascript6-native.js ./doxbee-sequential/promises-bluebird.js | sed -e 's|\.js||' | xargs node ./performance.js --p 1 --t 1 --n 100000 $ echo ./madeup-parallel/promises-ecmascript6-native.js ./madeup-parallel/promises-bluebird.js ./madeup-parallel/promises-native-async-await.js| sed -e 's|\.js||' | xargs node ./performance.js --p 25 --t 1 --n 30000
To reproduce them, you can clone this Github repository: https://github.com/petkaantonov/bluebird/.
The benchmarking tools are in the benchmark/ folder.
Node.js versions tested
For this benchmark, we will only compare LTS versions of Node.js (i.e. even numbers).
Node.js 6.17.1
This old version of Node.js was known for the poor performance of its native promises.
Promises managed with async/await were not yet available.
Node.js 8.17.0
This version is the first to take advantage of the new promise management with async/await.
Node.js 10.20.1
This version includes many performance enhancements to the promises.
For the first time, the native promises achieve performance comparable to Bluebird.
Node.js 12.16.3
This release includes a significant performance gain for promises managed with async/await. (See the v8 blog)
However, there is a general decrease in performances of native promises compared to Node.js 10.
Node.js 14.2.0
There is an improvement in performances of native promises in the latest version of Node.js.
Performance of sequential promises
We can see that since Node.js 12, sequential execution of promises is faster with async/await.
This suits us well because the use of async/await to manage promises is precisely designed for sequential sequences in order to get closer to synchronous programming.
const user = await fetch('/api/users/1');
const job = await fetch(`/api/jobs/${user. jobId}`);
const colleagues = await fetch(`/api/users/byJob/${job.id}`);
It is also to be noted that Bluebird always retains the smallest memory footprint.
Performance of parallel promises
For the parallel execution of promises, you can see that Bluebird has up to 4x better performances than native Node.js promises.
This can be explained by numerous performance optimizations made by the creator of the library: https://www.reaktor.com/blog/javascript-performance-fundamentals-make-bluebird-fast/
const userIds = [21, 42, 84, 168];
const promises = userIds.map(id => fetch(`/api/users/${id}`));
const users = await Bluebird.all(promises);
We therefore prefer to use Bluebird for parallel promises executions.
In addition, the library has many high-level methods to facilitate the management of parallel treatments like Bluebird.map which allows to limit the number of promises executed in parallel:
const users = await Bluebird.map(
userIds,
id => fetch(`/api/users/${id}`),
{ concurrency: 10 }
);
But what happened after Node.js 10?
If we take again the two graphs on the sequential and parallel performances, we can see that the release of Node.js 10 brought significant improvements in performances but then from Node.js 12 these performances have drastically decreased.
By doing more tests, we can see that this change occurs between versions 7.4 and 7.5 of v8, the Javascript engine of Node.js and Chrome.
I didn't find anything special about it, so I asked twitter, and this led to this new issue been filed in v8's bugtracker: https://bugs.chromium.org/p/v8/issues/detail?id=10550
EDIT: 10/07/2020
In the first version of this article, I noticed strange results where Node.js 10 had the best performances.
It turns out that it is because of the size of the memory pages which doubled in v8 between Node 8 and Node 10.
The benchmark tool had to iterate on each page to calculate the memory used, so it was at least twice as slow in the latest versions of Node.
Thanks to Camillo Bruni for the investigations!
Performance benchmarks have been updated by disabling memory calculation.
More informations here.
