Vector Databases

Learning Objectives

By the end of this session, you will be able to:

• Understand what a Vector Database is

• Learn why vector databases are needed in RAG systems

• Understand how vector databases store embeddings

• Explore similarity search inside vector databases

• Learn the difference between traditional databases and vector databases

• Understand indexing and retrieval concepts

• Explore popular vector database platforms

Introduction

In the previous sessions, we learned:

• What embeddings are

• How embeddings are generated

• How vector similarity search works

At this point, we have an important question:

Where do all these embeddings get stored?

Consider a medium-sized enterprise knowledge base.

It may contain:

• 100,000 documents

• Millions of chunks

• Millions of embeddings

A simple spreadsheet cannot handle this.

Neither can a traditional database efficiently perform semantic similarity searches across millions of vectors.

This challenge led to the creation of:

Vector Databases

Vector databases are one of the most important technologies behind:

• RAG systems

• AI assistants

• Enterprise search

• Recommendation engines

• Modern Generative AI applications

Without vector databases, large-scale semantic search would be impractical.

Why This Topic Matters

Imagine an organization stores:

5 Million Document Chunks

Each chunk has:

1536-Dimensional Embedding

Now a user asks:

What is our remote work policy?

The system must identify the most relevant chunks within milliseconds.

Searching every vector manually would be extremely slow.

A vector database solves this problem by:

Store Vectors
      +
Index Vectors
      +
Search Efficiently

This enables fast and scalable retrieval.

What Is a Vector Database?

A Vector Database is a specialized database designed to store, index, and search vector embeddings.

Unlike traditional databases that search using:

Exact Values

Vector databases search using:

Semantic Similarity

Think of it as:

Traditional Database
        ?
Find Exact Match

Vector Database
        ?
Find Similar Meaning

This distinction is critical.

Traditional Database Example

Suppose we have a customer table.

Query:

SELECT * FROM Customers
WHERE Name = 'Sarah'

The database finds:

Exact Match

This works well for structured data.

However, it struggles with semantic meaning.

Why Traditional Databases Are Not Enough

Consider:

Document:

Annual Leave Policy

User Query:

Vacation Policy

Traditional database:

No Exact Match

Semantic relationship:

Vacation
˜
Annual Leave

Traditional databases do not naturally understand this relationship.

Vector databases are specifically designed for this problem.

How Vector Databases Work

At a high level:

Documents
      ?
Embeddings
      ?
Vector Database
      ?
Similarity Search
      ?
Results

The vector database stores embeddings and makes them searchable.

What Is Stored in a Vector Database?

Typically, each record contains:

Original Text

Example:

Employees receive 24 annual leave days.

Embedding

Example:

[0.12, 0.87, -0.23, ...]

Metadata

Example:

{
  "department": "HR",
  "category": "Policy",
  "year": "2026"
}

Together:

Text
+
Embedding
+
Metadata

form a searchable record.

Example Record

{
  "id": "001",
  "text": "Employees receive 24 annual leave days.",
  "vector": [0.12, 0.87, -0.23, ...],
  "department": "HR"
}

This structure is common across most vector databases.

High-Level Architecture

Documents
      ?
Chunking
      ?
Embeddings
      ?
Vector Database
      ?
Retriever
      ?
LLM
      ?
Answer

The vector database acts as the knowledge retrieval layer.

How Search Works

Step 1:

User asks:

How much vacation time do employees receive?

Step 2:

Generate query embedding.

Question
      ?
Embedding

Step 3:

Vector database compares the query vector against stored vectors.

Step 4:

Similarity scores are calculated.

Step 5:

Top results are returned.

Workflow:

Question
      ?
Embedding
      ?
Vector Search
      ?
Top Matches
      ?
LLM

This happens in milliseconds.

Why Vector Databases Are Fast

Imagine:

10 Million Embeddings

Comparing every vector individually would be slow.

Vector databases solve this using specialized indexing algorithms.

These algorithms help find nearby vectors efficiently.

Benefits:

• Faster search

• Lower latency

• Better scalability

This makes large-scale semantic search practical.

Understanding Indexing

Indexing is similar to organizing books in a library.

Without organization:

Search Every Book

With organization:

Go Directly to Relevant Section

Vector databases create indexes that help locate similar vectors quickly.

Example Library Analogy

Traditional Library:

Search Every Shelf

Indexed Library:

Go Directly to Science Section

Vector databases apply the same principle to embeddings.

Common Vector Database Operations

Insert

Store new embeddings.

Example:

New Policy Document

Search

Find similar embeddings.

Example:

Vacation Policy

Update

Replace outdated information.

Example:

Updated HR Policy

Delete

Remove obsolete content.

Example:

Expired Document

These operations support dynamic knowledge bases.

Metadata Filtering

Vector databases often support metadata filtering.

Example:

Search:

Leave Policy

Filter:

Department = HR

Result:

Only HR Documents

This improves retrieval precision.

Example Metadata Query

Question:

What is the reimbursement policy?

Filter:

Department = Finance

The search is restricted to finance documents.

This is extremely useful in enterprise systems.

Vector Database vs Traditional Database

This comparison explains why vector databases became popular in AI applications.

Popular Vector Databases

Several vector databases are widely used today.

ChromaDB

Popular among developers and researchers.

Advantages:

• Open-source

• Easy setup

• Beginner-friendly

Common use cases:

• Learning projects

• Small to medium RAG applications

Pinecone

Managed cloud vector database.

Advantages:

• Fully managed

• Scalable

• Production-ready

Common use cases:

• Enterprise AI systems

• Large-scale applications

Weaviate

Open-source and cloud-supported.

Advantages:

• Strong metadata support

• Flexible architecture

Common use cases:

• Enterprise search

• Knowledge assistants

Qdrant

Modern vector database focused on performance.

Advantages:

• Fast retrieval

• Efficient filtering

Milvus

Designed for large-scale vector search.

Advantages:

• Highly scalable

• Enterprise deployments

Choosing a Vector Database

Selection depends on:

Project Size

Small project:

ChromaDB

Large enterprise:

Pinecone
Milvus
Weaviate

Infrastructure Preferences

Managed:

Pinecone

Self-hosted:

ChromaDB
Qdrant
Milvus

Budget

Cost considerations vary across platforms.

There is no single best choice.

Real-World Example: University Assistant

Knowledge Base:

Admission Policies
Scholarship Rules
Hostel Guidelines
Course Catalog

Process:

Documents
      ?
Embeddings
      ?
Vector Database
      ?
Student Questions
      ?
Relevant Information

The vector database becomes the university's semantic knowledge engine.

Real-World Example: HR Assistant

Knowledge Base:

Leave Policies
Benefits Guide
Travel Rules
Remote Work Policies

Employee asks:

Can I work from another city?

Vector search retrieves:

Remote Work Guidelines

The assistant generates a response using retrieved information.

Role of Vector Databases in RAG

Recall the complete RAG workflow:

Documents
      ?
Chunking
      ?
Embeddings
      ?
Vector Database
      ?
Similarity Search
      ?
Relevant Chunks
      ?
LLM
      ?
Answer

The vector database acts as the retrieval engine.

Without it:

No Efficient Semantic Search

Common Challenges

Large Data Volumes

Millions of vectors require efficient indexing.

Storage Costs

High-dimensional vectors consume space.

Retrieval Accuracy

Poor embeddings reduce search quality.

Metadata Management

Filters must remain accurate.

Updating Content

Knowledge bases change frequently.

Production systems must address these challenges.

Future of Vector Databases

Vector databases continue evolving.

Modern trends include:

• Hybrid search

• Multimodal retrieval

• Graph-enhanced retrieval

• Real-time indexing

• AI-native database architectures

These innovations are expanding the capabilities of AI systems.

Enterprise Architecture Example

Document Sources
        ?
Data Ingestion
        ?
Embeddings
        ?
Vector Database
        ?
Retriever
        ?
LLM
        ?
Answer

This architecture powers many enterprise AI assistants today.

.NET Perspective

Popular technologies include:

• Azure AI Search

• Semantic Kernel

• Azure OpenAI

• ASP.NET Core

Many .NET enterprise applications use Azure AI Search as both a search and vector storage solution.

Python Perspective

Popular tools include:

• ChromaDB

• Pinecone

• Weaviate

• Qdrant

• Milvus

• LangChain

• LlamaIndex

Python provides extensive support for vector database integration.

Assignment

Research Activity

Compare:

• ChromaDB

• Pinecone

• Weaviate

Identify:

• Features

• Advantages

• Limitations

• Ideal use cases

Design Exercise

Design a vector database architecture for:

University Knowledge Assistant

Include:

• Document sources

• Embedding generation

• Metadata fields

• Search workflow

Key Takeaways

• Vector databases are designed to store and search embeddings efficiently.

• They enable semantic search based on meaning rather than exact keywords.

• Traditional databases are not optimized for large-scale similarity search.

• Metadata filtering improves retrieval precision.

• Vector databases are a critical component of modern RAG systems.

• Popular platforms include ChromaDB, Pinecone, Weaviate, Qdrant, and Milvus.

• Nearly every production RAG system relies on a vector database.

What's Next?

In Session 23, we will explore:

Working with ChromaDB

You will learn how to create collections, store embeddings, perform similarity searches, manage metadata, and build your first end-to-end RAG workflow using ChromaDB.