Vector search is becoming part of many applications that use AI for recommendations, semantic search, document retrieval, question answering, and retrieval-augmented generation. Once an application starts storing embeddings, an architectural question usually appears: should vector search run directly from the main database, or should frequently accessed vectors be moved into a caching layer such as Amazon ElastiCache?
There is no single answer that works for every application.
A database and a cache solve different problems. Your primary database is normally responsible for durable storage, transactions, relationships, updates, and keeping the application's source of truth. A cache is designed to make frequently accessed data available with lower latency and to reduce repeated work against the primary system.
Vector search adds another dimension because similarity search can involve large numbers of vectors and can become expensive when the same data is searched repeatedly.
A practical architecture often keeps the database as the source of truth while using ElastiCache for workloads where vector retrieval needs predictable, low-latency access.
The important question is not simply whether ElastiCache is faster. The real question is whether the application's access pattern justifies separating durable vector storage from the retrieval path.
What Does Vector Search Actually Do?
Traditional database searches normally look for exact values or values that satisfy a condition.
For example, an application might search for:
category = "laptops"
price < 100000
brand = "Dell"
Vector search works differently.
An AI model can convert text, an image, a product description, or another piece of information into a numerical representation called an embedding. The embedding represents characteristics of the original data in a form that allows similarity calculations.
For example, these two queries may use completely different words:
"How do I reset my password?"
"Where can I change my account login credentials?"
A keyword search may not consider them closely related because the words are different. Vector search can identify that their meanings are similar because their embeddings are close in vector space.
A vector database or vector-enabled data store keeps these embeddings and uses a similarity algorithm to find the vectors closest to a query vector.
That makes vector search useful for applications such as:
AI-powered document search
Product recommendations
Semantic customer support search
Retrieval-augmented generation
Similar-image search
Content recommendations
Personalized search
Knowledge-base retrieval
The architectural problem begins when this search becomes part of a high-volume application.
Why Put Vectors in the Main Database?
Keeping vectors in the primary database is often the simplest architecture.
Suppose an application already stores documents like this:
Document
├── document_id
├── customer_id
├── title
├── content
├── category
├── updated_at
└── embedding
The database can keep both the business data and its corresponding embedding.
This creates a straightforward relationship between the searchable content and the vector.
When a document changes, the application can update its content and embedding in the same data system.
This is particularly useful when the application needs strong consistency between business data and vector data.
For example, consider a document-management system.
A user edits a document and changes its security classification from public to internal. The application may need to ensure that the document cannot immediately appear in searches available to public users.
If the document and its access-control information live in the same database, the application has a much easier consistency model to reason about.
The database can remain the authoritative source for:
Document content
Metadata
Ownership
Permissions
Tenant relationships
Status
Version information
Embeddings
This simplicity is a significant architectural advantage.
Why Move Vector Retrieval to ElastiCache?
The situation changes when the same vectors are searched repeatedly.
Imagine an application receiving thousands of search requests per second.
The same popular documents may participate in thousands of similarity searches.
If every request requires the primary database to perform the complete retrieval operation, the database becomes responsible for both durable application workloads and high-frequency AI retrieval workloads.
That creates resource contention.
The database may simultaneously be handling:
User transactions
+
Business queries
+
Document updates
+
Vector updates
+
Vector similarity searches
A caching layer can separate some of this workload.
A simplified architecture looks like this:
┌──────────────────┐
│ Application │
└────────┬─────────┘
│
Search request
│
▼
┌──────────────────┐
│ ElastiCache │
│ Vector Retrieval │
└────────┬─────────┘
│
Cache miss / refresh
│
▼
┌──────────────────┐
│ Primary Database │
│ Source of Truth │
└──────────────────┘
The important point is that ElastiCache does not have to replace the database.
It can serve as a retrieval layer for data that is expensive or frequently accessed.
The First Decision: Is Your Data Read-Heavy?
This is one of the most useful questions to ask before introducing a vector cache.
Suppose your system has:
10 million stored documents
2 million active documents
100,000 frequently accessed documents
millions of search requests per day
The entire dataset may not need to be equally fast.
A smaller set of vectors may receive most of the traffic.
This is where caching becomes interesting.
If the application's access pattern is highly concentrated, ElastiCache can keep frequently used vector data close to the application and reduce repeated database work.
On the other hand, if every request searches a completely different part of a large dataset, caching may provide less benefit.
Caching works best when the workload has some level of reuse.
The Second Decision: How Fresh Must the Search Results Be?
Freshness is one of the most important differences between a primary database and a cache.
Consider a product catalog.
A product description changes:
Old description
↓
New description
↓
New embedding generated
↓
Vector index updated
If the primary database contains the new embedding immediately but the cache still contains the old vector, the search layer may temporarily operate on stale information.
This may be acceptable for some applications.
For example, a recommendation system may tolerate a short delay between a product update and its appearance in semantic search.
It may not be acceptable for systems where permissions or sensitive information are involved.
A useful way to classify workloads is:
Requirement | Primary Database | ElastiCache |
|---|
Durable storage | Strong fit | Not the primary purpose |
Transactions | Strong fit | Limited |
Very frequent reads | Can work, but may become expensive | Strong fit |
Lowest possible retrieval latency | Depends on workload | Strong fit |
Strong source-of-truth requirement | Strong fit | Usually not |
Frequently changing business data | Strong fit | Requires invalidation/update strategy |
Hot vector retrieval | Possible | Strong fit |
Large transactional relationships | Strong fit | Poor fit |
Temporary or derived data | Possible | Strong fit |
The decision should therefore be based on the consistency requirements of the application, not only on latency.
When the Main Database Is the Better Choice
There are several situations where moving vectors to a cache adds unnecessary complexity.
The Vector Data Is Small
If the vector dataset is relatively small and query traffic is moderate, the database may already provide enough performance.
Adding another service introduces:
If the database handles the workload comfortably, there may be little reason to introduce another retrieval layer.
Vector Data Changes Frequently
Suppose an application continuously updates documents and generates new embeddings.
If vectors change every few seconds, maintaining a separate cache can become difficult.
The architecture may look like this:
Database update
↓
Embedding generation
↓
Vector update
↓
Cache invalidation
↓
Cache refresh
Every additional step creates another opportunity for temporary inconsistency.
If freshness is critical, keeping the vector representation closer to the source data may be easier to operate.
Transactions Matter
Some applications need vector data and business data to change together.
For example, imagine an application that creates a document, assigns it to a tenant, updates its access policy, and stores its embedding.
If those operations need transactional behavior, keeping related data in the same durable system can simplify the design.
A cache should not become the system responsible for maintaining transactional correctness.
Complex Filtering Is Required
Vector similarity is often only one part of a search request.
A real request may look conceptually like:
Find the most similar documents where:
tenant = current tenant
document_status = active
region = user's region
security_level <= user's permission
document_type = "contract"
The more business rules are involved, the more important it becomes to understand exactly where those filters are enforced.
A primary database may already have the relationships and metadata required to apply those rules correctly.
Moving only the vectors into a cache can leave the application responsible for reconstructing that context.
When ElastiCache Becomes More Attractive
ElastiCache becomes more interesting when vector retrieval has a very different workload from the rest of the application.
High Read Volume
Suppose the application receives a large number of semantic search requests while the underlying documents change relatively slowly.
That is a good candidate for separating retrieval from durable storage.
The database can handle:
Writes
Metadata
Transactions
Durability
while the retrieval layer handles:
Frequent vector searches
Hot data
Low-latency access
High read volume
This separation can make scaling easier because the two workloads no longer have to grow at exactly the same rate.
Hot Data
Not every document needs to be equally available at low latency.
For example, an AI assistant for an enterprise might have millions of internal documents but only a subset may be accessed frequently.
Popular documentation, frequently referenced policies, product manuals, and recent knowledge articles can receive much more traffic than older documents.
A cache can be used to keep the frequently accessed portion of the retrieval workload closer to the application.
Predictable Retrieval Performance Matters
Some applications have strict latency requirements.
An AI application may need to perform several operations within a single user request:
User question
↓
Create embedding
↓
Vector search
↓
Retrieve documents
↓
Build prompt
↓
Call model
↓
Generate answer
If vector retrieval consumes a large part of the request latency, improving that step can improve the overall user experience.
The important measurement is not simply average latency.
Measure:
p50 latency
p95 latency
p99 latency
Throughput
Cache hit rate
Database load
Vector index size
Search result freshness
Production decisions should be based on those measurements rather than assumptions.
A Hybrid Architecture Is Often the Practical Answer
In many real applications, the choice does not have to be database versus cache.
A hybrid architecture can provide a better separation of responsibilities.
The primary database remains authoritative:
Primary Database
┌─────────────────┐
│ Documents │
│ Metadata │
│ Permissions │
│ Embeddings │
│ Versions │
└────────┬────────┘
│
Update / Refresh
│
▼
┌─────────────────┐
│ ElastiCache │
│ Hot Vector Data │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Search Requests │
└─────────────────┘
This architecture creates a clear ownership model.
The database answers:
"What is the authoritative state of this document?"
ElastiCache answers:
"What vector data should be available quickly for retrieval?"
That distinction is important because it prevents the cache from becoming a second database that the application has to keep perfectly synchronized.
How the Synchronization Should Work
The application needs a clear process for moving vector changes into the retrieval layer.
A common pattern is:
Document Created
↓
Store Document
↓
Generate Embedding
↓
Store Embedding
↓
Publish Update Event
↓
Update ElastiCache
For an update:
Document Updated
↓
Generate New Embedding
↓
Persist New Version
↓
Invalidate or Replace Cached Vector
↓
Search Uses New Version
The exact implementation depends on the application's consistency requirements.
The important part is that synchronization should be explicit.
Do not rely on developers remembering to update the cache from every possible code path.
A centralized update process is easier to test and monitor.
Version Your Embeddings
Embedding models change over time.
Suppose an application originally generated embeddings using model version A.
Later, the team moves to model version B.
The new embeddings may not be directly compatible with the old retrieval index or may have different characteristics.
Instead of treating the vector as an anonymous piece of data, store enough information to identify its generation.
Conceptually:
document_id
embedding
embedding_model
embedding_version
updated_at
This makes migration much easier.
A cache can also use the embedding version as part of its key or metadata.
For example:
document-123:embedding:v2
This avoids accidentally mixing vectors generated by different models.
Do Not Let the Cache Become the Only Copy
One of the most common architectural mistakes is treating a cache as permanent storage.
A cache should generally be considered replaceable.
The application should be able to recover if:
The cache is cleared.
Nodes are replaced.
Data expires.
A deployment changes the cache configuration.
A cache entry becomes corrupted.
A region becomes unavailable.
The database should provide a path to rebuild the cache.
That means the application should have a process similar to:
Database
↓
Read authoritative vector data
↓
Populate cache
↓
Resume vector retrieval
If rebuilding the cache is impossible because the database no longer contains the required information, the architecture has effectively turned the cache into a primary data store.
That should be an intentional decision, not an accident.
Security and Multi-Tenant Applications
Multi-tenant AI applications need additional care.
Imagine a SaaS application with thousands of customers.
A vector search request should never retrieve vectors belonging to another tenant simply because those vectors happen to exist in the same retrieval system.
Tenant isolation should be part of the data model and request path.
For example:
Tenant A
↓
Tenant-aware search
↓
Tenant A vectors
Tenant B
↓
Tenant-aware search
↓
Tenant B vectors
Do not depend solely on the application remembering to filter results after retrieval.
The safer approach is to design the retrieval model so tenant boundaries are represented explicitly and consistently.
The same principle applies to permissions, document visibility, and security classifications.
What About Cost?
Cost should be evaluated using the complete workload, not simply the price of the cache.
A database-only architecture may require additional database capacity because vector search consumes CPU, memory, storage, or I/O resources.
A hybrid architecture adds ElastiCache costs but may allow the primary database to remain smaller or serve fewer expensive read operations.
A simplified cost comparison looks like this:
Database-only
Database
├── Transactions
├── Business queries
├── Vector writes
└── Vector searches
Hybrid
Database
├── Transactions
├── Business queries
└── Durable vector storage
ElastiCache
└── High-frequency vector retrieval
The correct question is:
"Does the additional cache infrastructure reduce enough database pressure or improve enough application performance to justify its cost and operational complexity?"
That should be answered using production measurements.
A Practical Decision Framework
Use the following questions when deciding where vector search belongs.
Choose the Main Database When
The database is generally the better starting point when:
Vector data is closely tied to transactional records.
Strong consistency is important.
Vector data changes frequently.
Search volume is moderate.
Complex relational filtering is required.
The dataset is manageable.
Operational simplicity is more important than maximum retrieval performance.
The existing database already handles the workload comfortably.
Consider ElastiCache When
A cache becomes more attractive when:
Vector reads are very frequent.
A significant portion of traffic targets hot data.
Low and predictable retrieval latency matters.
The primary database is becoming a bottleneck.
Vector data changes less frequently than it is searched.
The application can tolerate a controlled freshness window.
The retrieval workload needs to scale independently from transactional workloads.
Consider a Hybrid Design When
A hybrid model is usually worth evaluating when:
The database must remain the source of truth.
Vector search has high read traffic.
Only part of the vector dataset is hot.
Search latency is important.
The application can support asynchronous synchronization.
Cache rebuilds can be performed from durable data.
Example: Customer Support AI
Consider a customer-support assistant with 20 million historical documents.
The application stores:
Ticket content
Customer information
Agent notes
Product information
Permissions
Document metadata
Embeddings
Most customers repeatedly ask questions about a smaller set of current product documentation.
A database-only architecture may be perfectly acceptable at first.
As usage grows, however, the application may observe that:
80% of searches
↓
20% of documents
That access pattern creates an opportunity.
The database can remain responsible for durable document storage and permissions, while frequently accessed vector data can be served through ElastiCache.
The architecture does not need to move every document into the cache.
The goal is to optimize the workload that actually causes the bottleneck.
Common Mistakes
Moving Everything Into the Cache
More cached data does not automatically mean better performance.
Caching the entire vector dataset can increase memory requirements and operational cost without providing much benefit if most vectors are rarely requested.
Start with measured access patterns.
Ignoring Stale Data
A cache can return an older vector after the underlying document has changed.
Define how much staleness the application can tolerate before designing the synchronization strategy.
Using the Cache as the Source of Truth
If the application cannot reconstruct vector data after cache loss, the architecture is fragile.
Keep a durable source from which the retrieval layer can be rebuilt.
Forgetting Permissions
Vector similarity does not understand your application's business permissions by itself.
A document being semantically relevant does not mean the current user is allowed to see it.
Authorization must remain part of the retrieval architecture.
Mixing Embedding Versions
Old and new embedding models should not accidentally share the same vector namespace or retrieval assumptions.
Track embedding versions and plan migrations explicitly.
Measuring Only Average Latency
An average response time can hide serious tail-latency problems.
Measure p95 and p99 latency because those values often better represent what users experience during busy periods.
Advantages and Disadvantages
Using the Main Database
Advantages:
Simpler architecture
Stronger data ownership model
Easier consistency
Natural relationship with business metadata
Easier transactional workflows
Fewer infrastructure components
Disadvantages:
Vector searches can compete with transactional workloads
Scaling may require scaling the database
High read traffic can become expensive
Search performance may affect unrelated application workloads
Using ElastiCache
Advantages:
Separates hot retrieval from primary database workloads
Can provide low-latency access for suitable workloads
Allows retrieval capacity to scale independently
Useful for frequently accessed vector data
Can reduce repeated pressure on the primary database
Disadvantages:
Adds infrastructure
Requires synchronization
Creates possible freshness gaps
Requires cache recovery and rebuild procedures
Adds monitoring and operational complexity
Hybrid Architecture
Advantages:
Database remains the source of truth
Retrieval can be optimized independently
Hot data can be served quickly
Clear separation between durable storage and retrieval
Can scale different workloads independently
Disadvantages:
More components
More complicated data flow
Requires synchronization logic
Requires careful failure handling
Requires clear consistency rules
Production Checklist
Before moving vector search into ElastiCache, verify the following:
Measure current vector search latency.
Measure database CPU, memory, I/O, and query load.
Identify whether the workload is read-heavy.
Determine whether requests repeatedly access the same vectors.
Define the acceptable data freshness window.
Keep the database as the durable source of truth when appropriate.
Define how vector updates reach the cache.
Define cache invalidation behavior.
Track embedding model versions.
Design tenant isolation carefully.
Keep authorization checks in the retrieval path.
Test cache loss and rebuild procedures.
Monitor cache hit rate.
Monitor p95 and p99 search latency.
Monitor database load before and after introducing the cache.
Calculate the complete infrastructure cost.
Test behavior when the cache is unavailable.
Document whether stale search results are acceptable.
Test large-scale vector updates before production.
Summary
The decision between vector search in the main database and vector retrieval through ElastiCache is primarily an architecture and workload decision.
The main database is usually the right place for durable vector data when consistency, transactions, relationships, permissions, and operational simplicity are important.
ElastiCache becomes more useful when vector retrieval is read-heavy, latency-sensitive, and sufficiently independent from transactional workloads. It is particularly attractive when a relatively small portion of the vector data receives a large percentage of search traffic.
For many production applications, the most practical design is a hybrid architecture. The database remains the source of truth, while ElastiCache provides a fast retrieval layer for data that benefits from caching.
The key is to avoid introducing a cache simply because vector search is involved. First measure the workload. Identify the actual bottleneck, understand the required freshness, and determine whether the retrieval workload needs to scale independently from the rest of the application.
If those conditions exist, separating durable vector storage from high-frequency vector retrieval can make the architecture easier to scale without turning the cache into another system that must be treated as the application's source of truth.
Join the conversation! Your thoughts help the community grow.