Skip to main content

Multi-thread with nodejs

· 6 min read
Ayyaz Akhtar
Engineer @ Disney

cover

Node.js is a very powerful server-side javascript framework that excels in single-thread asynchronous I/O and is constantly updating with new features.

This guide is meant to be a resource to understand the multi-threading APIs and provide:

Synchronous vs Asynchronous​

Synchronous vs Asynchronous

  • Synchronous - Operations are expected to start and finish in a time span.
  • Asynchronous - Operations can be deferred till a later time.

Concurrency​

Concurrency

  • Concurrency - Multiple operations are handled at the same time.

Sequential vs Parallel​

Sequential vs Parallel

  • Sequential - Multiple operations are done in a linear fashion.
  • Parallel - Multiple operations are done at the same time.

Distributed​

Distributed

  • Distributed - Multiple operations are done independently on segmented resources.

Process vs Thread​

Process vs Thread

  • Process - A program can use single or multiple threads.
  • Thread - Code execution is invoked and uses less resources than a process.

Javascript Engine​

Javascript-Engine

  • Javascript Engine - Node.js process runs on top of the google chrome v8 engine.
    • Heap - Memory management of variables.
    • Call Stack - Code running on main thread.
    • Thread Pool - Async code pending to be executed.
    • Callback Queue - Async code ready to be executed.
    • Event Loop - Async code is pushed on to empty call stack.

Thread Blocking Operation​

Here is an example of a long running operation that will increment a counter variable from 'zero' to 'ten billion'.

The snippet of code will block all other operations on the thread since it is a 'synchronous' operation.

  • ex: web browser freezes after clicking a button.
  • ex: locking a database record when making an update.

Prerequisite​

  • node -v @12+
  • echo '{ "type": "module" }' > package.json
const blockingOperation = (operations = 10_000_000_000) => {
let counter = 0;

while (counter < operations) counter += 1;

console.log({ counter });

return counter;
};

(() => blockingOperation())();

time node index.js

{ counter: 10000000000 }
node index.js 7.80s user 0.04s system 99% cpu 7.849 total
  • Executing this program takes around ~8 seconds.
    • Running on a single CPU core (single-thread).
    • Sync tasks are blocking the thread.

Split Operation into Async Tasks​

Building on the example, we introduce Promises to invoke asynchronous operations.

In addition, the large operation will be split based on the number of CPU cores on the machine.

  • This chunking strategy will shorten the task's size of operations.
  • tasks = (operations / threads)

Async will allow the program to handle concurrent operations.

  • ex: web server accepting multiple API requests.
  • ex: video game accepting multiple clicks on a keyboard.
import { cpus } from 'os';

const getUTCSeconds = () => new Date().getUTCSeconds();

const blockingOperation = (operations) => new Promise((resolve) => {
let counter = 0;

while (counter < operations) counter += 1;

console.log({ counter, utcSecond: getUTCSeconds() });

resolve(counter);
});

const splitOperationIntoAsyncTasks = async () => {
const operations = 10_000_000_000;
const threads = cpus().length;
const tasks = (operations / threads);

const blockingOperations = new Array(threads).fill()
.map(() => blockingOperation(tasks))

const countOperations = await Promise
.all(blockingOperations);

const totalCount = countOperations
.reduce((sum, counter) => (sum + counter), 0);

console.log({ operations, threads, tasks, totalCount });

return totalCount;
};

(async () => splitOperationIntoAsyncTasks())();

time node index.js

{ counter: 1250000000, utcSecond: 21 }
{ counter: 1250000000, utcSecond: 22 }
{ counter: 1250000000, utcSecond: 22 }
{ counter: 1250000000, utcSecond: 23 }
{ counter: 1250000000, utcSecond: 23 }
{ counter: 1250000000, utcSecond: 24 }
{ counter: 1250000000, utcSecond: 24 }
{ counter: 1250000000, utcSecond: 25 }
{
operations: 10000000000,
threads: 8,
tasks: 1250000000,
totalCount: 10000000000
}
node index.js 4.35s user 0.02s system 99% cpu 4.372 total
  • Executing this program takes around ~4 seconds.
  • We see a 50% performance improvement.
    • Running on a single CPU core (single-thread).
    • Async tasks are processed in sequential.

Invoke Multi Thread Operations​

The final step is to introduce the multi-thread API worker threads.

There are two concepts that are at play:

  • Parent (main thread)
    • if (isMainThread) return new Worker(__filename)
  • Child (worker thread)
    • parentPort.postMessage(await blockingOperation())

The parent will create multiple children and wait for the messages to be returned.

import { cpus } from 'os';
import { dirname } from 'path';
import { fileURLToPath } from 'url';
import { isMainThread, Worker, parentPort } from 'node:worker_threads';

const __filename = dirname(fileURLToPath(import.meta.url));

const createWorker = async () => new Promise ((resolve, reject) =>
new Worker(__filename)
.on('message', (data) => resolve(data))
.on('error', (error) => reject(error))
.on('exit', (code) => { if (code !== 0) reject(new Error(`Thread exit code: ${code}`)); }));

const getUTCSeconds = () => new Date().getUTCSeconds();

const blockingOperation = (operations) => new Promise((resolve) => {
let counter = 0;

while (counter < operations) counter += 1;

console.log({ counter, utcSecond: getUTCSeconds() });

resolve(counter);
});

const runTaskInParallel = async (operations, threads, tasks) => {
const blockingOperations = new Array(threads).fill()
.map(() => createWorker(tasks));

const countOperations = await Promise
.all(blockingOperations);

const totalCount = countOperations
.reduce((sum, counter) => (sum + counter), 0);

console.log({ operations, threads, tasks, totalCount });

return totalCount;
};

const invokeMultiThreadOperations = async () => {
const operations = 10_000_000_000;
const threads = cpus().length;
const tasks = (operations / threads);

if (isMainThread) return runTaskInParallel(operations, threads, tasks);

parentPort.postMessage(await blockingOperation(tasks));
};

(async () => invokeMultiThreadOperations())();

time node index.js

{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{ counter: 1250000000, utcSecond: 59 }
{
operations: 10000000000,
threads: 8,
tasks: 1250000000,
totalCount: 10000000000
}
node index.js 6.38s user 0.05s system 711% cpu 0.903 total
  • Executing this program takes around ~1 second.
  • We see a 87% performance improvement.
    • Running on a multi CPU core (multi-thread).
    • Async tasks are processed in parallel.