1. Introduction 🧵

JavaScript in Node runs on a single thread, but real applications often need true parallelism — running another program, offloading heavy computation, or scaling across CPU cores. Node provides three complementary tools for this: child_process, worker_threads, and cluster.

2. Understanding Concurrency in Node.js 🔀

Concurrency (handling many things at once via the event loop) is different from parallelism (actually executing code simultaneously on multiple cores). Node's event loop gives you concurrency for free; child_process and worker_threads are how you get real parallelism.

3. Child Processes 👶

The child_process module (node:child_process) lets Node spawn and control other operating system processes — including shell commands, executables, or other Node scripts.

3.1 spawn()

spawn() launches a command with streamed input/output — ideal for long-running processes or large output volumes.

const { spawn } = require('node:child_process');

const child = spawn('ls', ['-la']);

child.stdout.on('data', (data) => console.log(`Output: ${data}`));
child.stderr.on('data', (data) => console.error(`Error: ${data}`));
child.on('close', (code) => console.log(`Exited with code ${code}`));

3.2 exec()

exec() runs a command in a shell, buffering the entire output and returning it via a callback — convenient for short commands with small output.

const { exec } = require('node:child_process');

exec('ls -la', (err, stdout, stderr) => {
  if (err) return console.error(err);
  console.log(stdout);
});

3.3 execFile()

execFile() runs an executable directly, without a shell — safer and slightly more efficient than exec() when you don't need shell features like pipes or globbing.

const { execFile } = require('node:child_process');

execFile('node', ['--version'], (err, stdout) => {
  if (err) return console.error(err);
  console.log(stdout);
});

3.4 fork()

fork() is a specialized spawn() for launching another Node.js module as a child process, automatically setting up an IPC channel for message passing.

const { fork } = require('node:child_process');

const child = fork('./worker-script.js');

child.on('message', (msg) => console.log('From child:', msg));
child.send({ task: 'process', data: [1, 2, 3] });

4. IPC (Inter-Process Communication) 📨

When using fork(), parent and child communicate via .send() and the 'message' event — a built-in IPC channel that serializes data as JSON-like messages.

// worker-script.js
process.on('message', (msg) => {
  const result = msg.data.reduce((a, b) => a + b, 0);
  process.send({ result });
});

5. Standard Input & Output 🔌

Every child process (from spawn or fork) exposes stdin, stdout, and stderr as streams, letting the parent pipe data in and read output out.

const { spawn } = require('node:child_process');

const grep = spawn('grep', ['error']);

grep.stdin.write('info: starting\n');
grep.stdin.write('error: connection failed\n');
grep.stdin.end();

grep.stdout.on('data', (data) => console.log(data.toString()));

6. Process Management 🎛️

const child = spawn('long-running-task');

child.kill('SIGTERM'); // request graceful termination
console.log(child.pid); // process ID
console.log(child.killed); // whether kill() was called

7. Worker Threads 🧶

The worker_threads module runs JavaScript in parallel threads within the same process — each with its own V8 instance and event loop, but able to share memory via SharedArrayBuffer. Lighter weight than spawning a whole new process.

7.1 Creating Worker Threads

const { Worker, isMainThread, parentPort, workerData } = require('node:worker_threads');

if (isMainThread) {
  const worker = new Worker(__filename, { workerData: { num: 42 } });
  worker.on('message', (result) => console.log('Result:', result));
} else {
  const squared = workerData.num ** 2;
  parentPort.postMessage(squared);
}

7.2 Message Passing

Communication between the main thread and a worker happens via postMessage(), which structured-clones the data (copying it) rather than sharing memory directly.

worker.postMessage({ type: 'compute', payload: [1, 2, 3] });

worker.on('message', (msg) => console.log('Worker replied:', msg));

8. Shared Memory 🧠

8.1 SharedArrayBuffer

Unlike regular message passing (which copies data), a SharedArrayBuffer is actual shared memory that multiple threads can read and write directly, without serialization overhead.

const sharedBuffer = new SharedArrayBuffer(4);
const sharedArray = new Int32Array(sharedBuffer);

worker.postMessage({ sharedBuffer });

8.2 Atomics

Atomics provides thread-safe operations on shared memory — reading, writing, and waiting — that are guaranteed not to be interrupted mid-operation by another thread.

Atomics.add(sharedArray, 0, 1);      // atomic increment
Atomics.store(sharedArray, 0, 42);   // atomic write
const value = Atomics.load(sharedArray, 0); // atomic read

9. Thread Pools 🏊

Rather than spinning up a new Worker per task (which has real overhead), production systems often maintain a pool of reusable worker threads, distributing tasks among them.

const { Worker } = require('node:worker_threads');

class WorkerPool {
  constructor(size, script) {
    this.workers = Array.from({ length: size }, () => new Worker(script));
    this.nextWorker = 0;
  }

  run(data) {
    const worker = this.workers[this.nextWorker];
    this.nextWorker = (this.nextWorker + 1) % this.workers.length;
    return new Promise((resolve) => {
      worker.once('message', resolve);
      worker.postMessage(data);
    });
  }
}

10. CPU-Intensive Tasks 🔢

Tasks like image processing, cryptographic hashing, or complex calculations block the event loop if run synchronously on the main thread. Offloading them to worker_threads keeps the server responsive.

// fibonacci-worker.js
const { parentPort, workerData } = require('node:worker_threads');

function fib(n) {
  return n <= 1 ? n : fib(n - 1) + fib(n - 2);
}

parentPort.postMessage(fib(workerData.n));

11. Background Jobs 🕰️

Longer background work (sending bulk emails, generating reports, processing uploads) is often delegated to a child process or an external job queue (like BullMQ backed by Redis) so it doesn't compete with request handling.

12. Clustering Overview 🌐

The cluster module lets you fork multiple Node processes that share the same server port, distributing incoming connections across them — a simple way to use all CPU cores for an HTTP server.

const cluster = require('node:cluster');
const os = require('node:os');
const http = require('node:http');

if (cluster.isPrimary) {
  const cpuCount = os.cpus().length;
  for (let i = 0; i < cpuCount; i++) cluster.fork();
} else {
  http.createServer((req, res) => res.end('Handled by worker ' + process.pid)).listen(3000);
}

13. Process vs Thread vs Cluster 🆚

14. Choosing the Right Approach 🧭

15. Performance Optimization ⚡

16. Best Practices ✅

17. Common Mistakes ⚠️

18. Frequently Asked Questions ❓

19. Summary 📝