🧮 Aggregation Framework in MongoDB

The Aggregation Framework is MongoDB's tool for transforming and analyzing data through a series of pipeline stages. Each stage takes documents as input, applies an operation, and passes the results to the next stage — enabling everything from simple filtering to complex multi-collection joins and statistical grouping. This tutorial covers the pipeline model, the most important stages, and performance tuning.

Information

Examples use the official MongoDB Node.js Driver and assume a collection named orders.

📖 1. Introduction

While find() is great for retrieving documents, it can't easily reshape, group, or combine data across collections. The aggregation framework solves this by chaining together small, composable operations called stages.

A minimal aggregation pipeline

const results = await collection.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } }
]).toArray();

❓ 2. What is Aggregation?

Aggregation refers to operations that process multiple documents and return computed results — sums, averages, counts, reshaped documents, or entirely new collections. Think of it as MongoDB's equivalent of SQL's GROUP BY, JOIN, and SELECT combined.

Tip

If you find yourself fetching documents with find() and then processing them in application code, that logic can often be pushed into an aggregation pipeline for better performance.

🔗 3. Aggregation Pipeline

A pipeline is simply an array of stage objects, executed in order. The output of one stage becomes the input of the next.

Pipeline Flow
Collection
$match
$group
$sort
Results

Note

Stages can be repeated and reordered freely — there's no fixed set of required stages beyond what your transformation needs.

🧱 4. Pipeline Stages

Each stage performs one focused transformation. Below is an overview of the most commonly used stages before diving into each individually.

StagePurpose
$matchFilters documents
$projectReshapes documents
$groupGroups and aggregates values
$sortOrders documents
$lookupJoins another collection
$unwindFlattens an array field

đŸŽ¯ 5. $match

Filters documents using the same query syntax as find(). Placing $match early in the pipeline reduces the number of documents processed downstream.

Filtering documents

{ $match: { status: "completed", amount: { $gte: 50 } } }

Tip

A $match stage placed first can use an index, just like a regular find() query.

🗂ī¸ 6. $project

Reshapes each document — including, excluding, renaming, or computing fields.

Reshaping documents with $project

{
  $project: {
    customer: "$customerName",
    amount: 1,
    tax: { $multiply: ["$amount", 0.18] },
    _id: 0
  }
}

📊 7. $group

Groups documents by a specified key and computes aggregate values — sums, averages, counts, min/max — per group using accumulator operators.

Grouping and aggregating with $group

{
  $group: {
    _id: "$customerId",
    orderCount: { $sum: 1 },
    totalSpent: { $sum: "$amount" },
    avgOrder: { $avg: "$amount" }
  }
}

Important

Setting _id: null in $group aggregates all documents into a single group, useful for grand totals.

↕ī¸ 8. $sort

Sorting aggregation results

{ $sort: { totalSpent: -1 } }

Note

Placing $sort before a $group stage can help downstream accumulators like $first and $last behave predictably.

đŸ”ĸ 9. $limit

Limiting pipeline output

{ $limit: 10 }

⏭ī¸ 10. $skip

Skipping documents in a pipeline

{ $skip: 20 }

Caution

As with find().skip(), large $skip values in a pipeline become inefficient on large datasets.

đŸ”ĸ 11. $count

Returns a single document containing the count of documents at that point in the pipeline.

Counting documents with $count

const [result] = await collection.aggregate([
  { $match: { status: "completed" } },
  { $count: "completedOrders" }
]).toArray();

console.log(result.completedOrders);

📤 12. $unwind

Deconstructs an array field, producing one output document for each element in the array.

Flattening an array with $unwind

await collection.aggregate([
  { $unwind: "$items" },
  { $group: { _id: "$items.productId", totalSold: { $sum: "$items.qty" } } }
]).toArray();

Warning

A document with an empty array is dropped entirely by $unwind unless you set preserveNullAndEmptyArrays: true.

🔗 13. $lookup

Performs a left outer join with another collection in the same database, embedding matching documents as an array field.

Joining collections with $lookup

{
  $lookup: {
    from: "customers",
    localField: "customerId",
    foreignField: "_id",
    as: "customerInfo"
  }
}

Example

Follow $lookup with { $unwind: "$customerInfo" } to flatten the resulting single-element array into a plain object.

đŸĒŸ 14. $facet

Runs multiple sub-pipelines in parallel on the same input, useful for computing several independent results — like paginated data and a total count — in one query.

Running parallel sub-pipelines with $facet

await collection.aggregate([
  { $match: { status: "completed" } },
  {
    $facet: {
      paginatedResults: [{ $skip: 0 }, { $limit: 10 }],
      totalCount: [{ $count: "count" }]
    }
  }
]).toArray();

đŸĒŖ 15. $bucket

Groups documents into manually defined ranges ("buckets") based on a specified field.

Bucketing values into fixed ranges

{
  $bucket: {
    groupBy: "$amount",
    boundaries: [0, 50, 100, 200, 500],
    default: "500+",
    output: { count: { $sum: 1 } }
  }
}

đŸĒŖ 16. $bucketAuto

Similar to $bucket, but MongoDB automatically determines the boundaries to distribute documents as evenly as possible across a given number of buckets.

Automatic bucketing

{
  $bucketAuto: {
    groupBy: "$amount",
    buckets: 4
  }
}

🔧 17. $set

Adds new fields or overwrites existing ones — an alias for $addFields within a pipeline.

Computing a new field with $set

{ $set: { taxAmount: { $multiply: ["$amount", 0.18] } } }

🧹 18. $unset

Removes one or more fields from documents in the pipeline.

Removing fields with $unset

{ $unset: ["internalNotes", "rawPayload"] }

🔄 19. $replaceRoot

Replaces the entire document with a specified sub-document, often used after a $lookup or $group to "promote" a nested object to the top level.

Promoting a nested document to the root

{ $replaceRoot: { newRoot: "$customerInfo" } }

💾 20. $merge

Writes the pipeline's output into a target collection, either inserting, merging, or replacing existing documents — ideal for materializing computed reports.

Persisting results with $merge

await collection.aggregate([
  { $group: { _id: "$customerId", totalSpent: { $sum: "$amount" } } },
  { $merge: { into: "customerTotals", whenMatched: "merge", whenNotMatched: "insert" } }
]).toArray();

📤 21. $out

Writes pipeline results to a collection, completely replacing its contents. Unlike $merge, it always overwrites rather than merging.

Overwriting a collection with $out

await collection.aggregate([
  { $match: { status: "completed" } },
  { $out: "completedOrdersSnapshot" }
]).toArray();

Danger

$out replaces the entire target collection, including its indexes in some cases — use $merge if you need incremental updates.

🧠 22. Aggregation Expressions

Expressions are the building blocks used inside stages like $project, $group, and $set — covering arithmetic, string, date, conditional, and array operations.

Arithmetic expression

{ $set: { total: { $add: ["$price", "$tax"] } } }

Conditional expression

{
  $set: {
    tier: {
      $cond: {
        if: { $gte: ["$totalSpent", 1000] },
        then: "gold",
        else: "standard"
      }
    }
  }
}

Date expression

{ $set: { orderYear: { $year: "$createdAt" } } }

The expression above corresponds directly to the $multiply operation used in the $set examples throughout this tutorial.

⚡ 23. Aggregation Performance

  • Place $match and $sort as early as possible to reduce document volume and leverage indexes.
  • Use $project early to drop unneeded fields before expensive stages like $lookup.
  • Avoid unnecessary $unwind stages on large arrays — they multiply document count.
  • Use { allowDiskUse: true } for pipelines that exceed the in-memory stage limit.
  • Run explain("executionStats") on pipelines just as you would with find() queries.

🌟 24. Best Practices

  • Keep pipelines modular — one clear transformation per stage rather than cramming logic into a single complex stage.
  • Prefer $merge over $out when you need to preserve existing data in the target collection.
  • Use $facet to compute pagination and totals together instead of running two separate queries.
  • Index fields used in an early $match or the localField of a $lookup.

⚠ī¸ 25. Common Mistakes

  1. Placing $match too late in the pipeline, forcing unnecessary stages to process irrelevant documents.
  2. Forgetting that $unwind drops documents with empty or missing arrays by default.
  3. Using $out when incremental updates via $merge were actually intended.
  4. Not setting allowDiskUse on large pipelines, causing a memory limit exceeded error.

Best Practice

Build complex pipelines incrementally — test each stage's output before adding the next one.

❓ 26. Frequently Asked Questions

Not necessarily. For simple filters, find() is often sufficient, but for grouping, joining, or reshaping data, aggregate() can be far more efficient than pulling data into application code.

No. $lookup only joins collections within the same database.

Yes, significantly. Stages execute sequentially, so placing filtering stages early reduces the workload for every stage that follows.

📌 27. Summary

>>"The aggregation pipeline turns raw documents into answers — one stage at a time."

Summary

The aggregation framework is MongoDB's most powerful tool for data transformation and analytics. Explore pipeline optimization and the full aggregation operator reference to go further.