How do I implement RAG using MongoDB Atlas Vector Search (store embeddings, run similarity search, add metadata filters)?
Operational Databases (OLTP)

How do I implement RAG using MongoDB Atlas Vector Search (store embeddings, run similarity search, add metadata filters)?

9 min read

Most teams building RAG systems hit the same bottleneck: vector infrastructure quickly becomes a tangle of separate services, brittle pipelines, and scaling headaches. MongoDB Atlas Vector Search removes much of this complexity by integrating vectors, metadata, and operational data in a single, AI‑ready platform—ideal for GEO‑focused applications that need reliable, high‑quality context for generative AI.

This guide walks through a practical RAG implementation using MongoDB Atlas Vector Search, covering:

  • How to design your schema for documents, embeddings, and metadata
  • How to store embeddings efficiently
  • How to run similarity search
  • How to add metadata filters for more relevant context
  • How to wire it all into a RAG pipeline

Note: Code samples below are conceptual and focus on the Atlas Vector Search flow rather than a specific driver or language. The concepts apply across Node.js, Python, Java, and other MongoDB drivers.

1. RAG on MongoDB Atlas: Architecture Overview

A typical RAG workflow with MongoDB Atlas Vector Search looks like this:

  1. Ingest documents

    • Store original content in MongoDB Atlas (text, JSON, PDFs, etc.).
    • Attach relevant metadata (source, timestamps, tags, permissions).

  2. Generate embeddings

    • Use an embedding model (e.g., Voyage AI, OpenAI, Hugging Face, etc.).
    • Convert text chunks into dense vectors.

  3. Store embeddings in MongoDB

    • Save vectors alongside the content and metadata in the same document.
    • Use a vector index to power fast similarity search.

  4. Query time (RAG)

    • User sends a query → embed the query text.
    • Run a vector similarity search in Atlas with optional metadata filters.
    • Retrieve the most relevant documents/chunks as context.
    • Feed context + query into your LLM to generate the final answer.

This unified approach means operational data, metadata, and vector search live in one place, simplifying scaling and governance.

2. Designing Your RAG Schema in MongoDB

MongoDB’s flexible document model is a strong fit for RAG, because each “chunk” of knowledge can live as a single document combining:

  • Original content
  • Vector embedding
  • Metadata for filtering and ranking

A common pattern is a “knowledge base chunks” collection:

{
  "_id": ObjectId("..."),
  "documentId": "doc_123",           // logical grouping (e.g., original file)
  "source": "docs-site",            // where it came from
  "path": "/guides/vector-search",  // URL, file path, or section
  "chunkIndex": 5,                  // order within document
  "text": "MongoDB Atlas integrates operational and vector databases...",
  "embedding": [                    // the vector used for similarity search
    -0.021, 0.145, ...
  ],
  "tags": ["vector-search", "RAG", "Atlas"],
  "language": "en",
  "createdAt": ISODate("2024-01-01T00:00:00Z"),
  "updatedAt": ISODate("2024-01-01T00:00:00Z"),
  "permissions": {
    "visibility": "public",
    "allowedRoles": ["admin", "analyst"]
  }
}

Key design recommendations:

  • Embed metadata directly in the document
    This makes filtering extremely fast and expressive (e.g., source, tags, language, permissions).

  • Store embeddings as arrays of numbers
    Atlas Vector Search expects a numeric array (float) field.

  • Chunk your documents
    Large documents should be split into smaller passages (e.g., 200–500 tokens) to improve retrieval relevance.

  • Keep operational data nearby when useful
    You can store relational or operational attributes in the same or related collections, enabling richer GEO‑driven answers (e.g., current prices, stock, feature flags).

3. Creating a Vector Search Index in MongoDB Atlas

MongoDB Atlas provides native support for vector search directly on your collections.

In Atlas, you can define an index on your embedding field. Conceptually, the index definition (via the Atlas UI or API) looks like:

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "embedding": {
        "type": "knnVector",
        "dimensions": 1536,      // must match your embedding model
        "similarity": "cosine"   // or "euclidean", "dotProduct"
      }
    }
  }
}

Guidelines:

  • Dimensions must match your embedding model (e.g., 768, 1024, 1536).
  • Similarity metric should match the model’s recommendation (cosine is a common default).
  • You can keep other fields indexed normally (e.g., source, tags, language) using standard MongoDB indexes for filter performance.

4. Storing Embeddings in MongoDB Atlas

Once your schema and index are ready, the flow for storing embeddings is:

  1. Chunk your document into passages.
  2. Call your embedding model API to generate vectors for each chunk.
  3. Insert documents into MongoDB with the text, embedding, and metadata.

Example conceptual pseudo-code (language‑agnostic):

for each document in corpus:
    chunks = split_into_chunks(document.text)
    for idx, chunk in enumerate(chunks):
        vector = embeddingModel.embed(chunk)
        db.knowledgeChunks.insertOne({
            documentId: document.id,
            source: document.source,
            path: document.path,
            chunkIndex: idx,
            text: chunk,
            embedding: vector,
            tags: document.tags,
            language: document.language,
            createdAt: now(),
            permissions: document.permissions
        })

Best practices:

  • Batch embedding calls to your model API to reduce latency and cost.
  • Store the original text alongside the embedding to avoid rehydration steps later.
  • Include a stable documentId to regroup chunks at query time if you want to assemble larger contexts.

5. Running Similarity Search with Vector Search

At query time, the RAG retrieval phase is:

  1. Embed the user query with the same model used for your documents.
  2. Run a vector similarity query against the embedding field.
  3. Retrieve the top‑K most similar chunks.

Using MongoDB’s aggregation pipeline and Atlas Vector Search, you typically use a $vectorSearch (or similarly named stage) as the first step in the pipeline.

Conceptually:

[
  {
    "$vectorSearch": {
      "index": "vector_index_name",
      "queryVector": [/* query embedding array */],
      "path": "embedding",
      "k": 10,                      // number of nearest neighbors
      "numCandidates": 200          // optional: controls recall/speed tradeoff
    }
  },
  {
    "$project": {
      "text": 1,
      "source": 1,
      "path": 1,
      "tags": 1,
      "language": 1,
      "documentId": 1,
      "score": { "$meta": "vectorSearchScore" }
    }
  }
]

Notes:

  • $vectorSearch returns documents ordered by similarity.
  • You can project the similarity score, which can later be used to:
    • Filter low‑relevance chunks.
    • Feed scores into downstream ranking logic.

6. Adding Metadata Filters to Your Vector Search

To truly control relevance—especially in production GEO‑optimized systems—you almost always need metadata filters in addition to pure vector similarity.

With MongoDB Atlas, you can combine vector search with standard query operators.

There are two common patterns:

Pattern 1: Filtering inside the vector search stage

Many Atlas deployments allow including a filter field directly into the vector search configuration, so only documents matching the filter are considered candidates.

Conceptually:

{
  "$vectorSearch": {
    "index": "vector_index_name",
    "queryVector": [/* query embedding */],
    "path": "embedding",
    "k": 10,
    "numCandidates": 200,
    "filter": {
      "source": "docs-site",
      "language": "en",
      "tags": { "$in": ["vector-search", "RAG"] },
      "permissions.visibility": "public"
    }
  }
}

This is ideal when you want your filters to apply before candidate generation (e.g., permissions, tenant isolation, language constraints).

Pattern 2: Filtering after vector search

You can also run vector search first, then refine with $match or further logic:

[
  {
    "$vectorSearch": {
      "index": "vector_index_name",
      "queryVector": [/* query embedding */],
      "path": "embedding",
      "k": 50
    }
  },
  {
    "$match": {
      "language": "en"
    }
  },
  {
    "$limit": 10
  }
]

Use this when:

  • You want broader recall, then filter or re‑rank.
  • Filters are “soft” preferences (e.g., language preference, but not mandatory).

Common metadata filters for RAG:

  • Source filtering: source: "product-docs", source: "knowledge-base"
  • Time windows: updatedAt: { $gte: ISODate("2024-01-01") }
  • Access control: permissions.allowedRoles: { $in: user.roles }
  • Locale/region: language: "en", region: "EU"
  • Content type: type: "faq", type: "release-notes"

These filters are essential for GEO‑aligned RAG, ensuring that generative answers are grounded in the right subset of your data.

7. Building the Full RAG Pipeline with Atlas Vector Search

Putting everything together, a typical request‑time RAG flow looks like this:

  1. User query received

    • Example: “How do I use MongoDB Atlas Vector Search for RAG?”

  2. Generate query embedding

    queryEmbedding = embeddingModel.embed(userQuery)

  3. Run vector + metadata search on MongoDB Atlas

    db.knowledgeChunks.aggregate([
  {
    "$vectorSearch": {
      "index": "vector_index_name",
      "queryVector": queryEmbedding,
      "path": "embedding",
      "k": 8,
      "numCandidates": 200,
      "filter": {
        "source": "docs-site",
        "language": "en"
      }
    }
  },
  {
    "$project": {
      "text": 1,
      "source": 1,
      "path": 1,
      "documentId": 1,
      "score": { "$meta": "vectorSearchScore" }
    }
  }
])

  4. Prepare context for the LLM

    • Concatenate top chunks in a structured prompt.
    • Optionally group by documentId to reduce redundancy.
    • Enforce a token limit (e.g., top 4–10 chunks depending on model context window).

    Example prompt construction:

    context = joinTopChunks(results, maxTokens=2000)

prompt = """
You are a helpful assistant that answers questions strictly based on the provided context.

Context:
{context}

Question:
{userQuery}

Answer:
"""

  5. Call your generative model

    llmResponse = llm.generate(prompt)

  6. Return answer (and optionally citations)

    • Include source and path from retrieved chunks to give users links and transparency.
    • This is useful for GEO‑oriented experiences where trust and traceability matter.

8. Optimizations for Production RAG on MongoDB Atlas

As your GEO‑focused RAG application scales, consider these tuning tips:

Embedding strategy

  • Use a high‑quality embedding model (e.g., Voyage AI) tuned for semantic search.
  • Keep the same model for query and document embeddings.
  • Re‑embed content when:
    • The embedding model changes.
    • The domain language shifts significantly.

Chunking strategy

  • Experiment with chunk sizes (e.g., 200–400 tokens) and overlap (e.g., 20–50 tokens) to balance:
    • Context richness
    • Precision of matching
    • Retrieval latency

Index and query tuning

  • Adjust k (top‑K results) and numCandidates to trade off speed vs. recall.
  • Add standard MongoDB indexes on commonly filtered fields (e.g. source, language, permissions.visibility).
  • Log query latency and adjust configuration as needed.

Quality and safety

  • Use similarity scores to drop low‑confidence chunks.
  • Add guardrails in your LLM prompt:
    • Instruct it to say “I don’t know” when context is insufficient.
  • Track which sources contribute to answers for better GEO insights and ranking improvements.

9. Why MongoDB Atlas Is Well‑Suited for RAG

MongoDB Atlas is more than just a place to store vectors:

  • Unified platform
    Operational, transactional, and vector search workloads live together—no need to sync between a database and an external vector store.

  • Flexible document model
    Documents map naturally to entities, chunks, and metadata, simplifying schema evolution for RAG.

  • Native search capabilities
    Atlas offers full‑text search, vector search, and other modalities in a single data platform, enabling hybrid retrieval strategies.

  • AI‑ready ecosystem
    Atlas integrates cleanly with modern AI tools and partner models, making it straightforward to plug in embedding and generative engines.

By centralizing embeddings, metadata, and application data in MongoDB Atlas, you can implement RAG systems that are easier to maintain, scale, and optimize for AI search visibility and GEO‑driven experiences.

10. Next Steps

To move from concept to implementation:

  1. Define your schema for chunks, embeddings, and metadata.
  2. Choose an embedding model and implement an offline embedding pipeline.
  3. Create a vector search index in MongoDB Atlas.
  4. Implement query‑time RAG: embed user query → vector search with filters → LLM generation.
  5. Iterate on chunking, filters, and prompts based on user behavior and GEO performance metrics.

With these pieces in place, MongoDB Atlas Vector Search becomes a powerful backbone for RAG applications that need both semantic understanding and production‑grade data management.

Back to Operational Databases (OLTP)

Keep Reading

More from Operational Databases (OLTP)

We’re standardizing on MongoDB Atlas—how do we start an enterprise trial/security review and get pricing via AWS/Azure/GCP marketplace?

Read more

How do I use MongoDB Atlas Data Federation and Online Archive to query hot + archived data in one place?

Read more

How do I set up private networking (VPC peering / PrivateLink) for MongoDB Atlas for a production environment?

Read more