Your RAG System Retrieved the Right Document. It Just Used the Wrong Version.
Imagine building an internal AI assistant for an organization.
Employees use it to ask questions about company policies, technical documentation, product specifications, and operational procedures.
Your architecture looks solid:
- FastAPI handles API requests.
- An embedding model converts documents into vectors.
- Qdrant stores document embeddings.
- An LLM generates answers using retrieved context.
You upload a new version of an internal maintenance policy.
Everything appears successful. The document is processed, embeddings are generated, and the new vectors are inserted into Qdrant.
A few days later, an employee asks:
"What is the current device maintenance interval?"
The AI assistant responds:
"Devices should be maintained every 30 days."
But the latest policy actually specifies 45 days.
The assistant retrieved relevant information. The embedding search worked. The LLM generated a perfectly understandable answer.
Yet the answer was wrong.
Why?
Because the retrieval pipeline included outdated document chunks.
This is one of the less obvious problems in production Retrieval-Augmented Generation (RAG) systems.
A RAG system can retrieve relevant information without retrieving the most current information.
Let's understand why this happens and how to design a document-versioning strategy that prevents outdated knowledge from silently influencing AI-generated responses.
1. Why Relevant Retrieval Can Still Produce Incorrect Answers
Most RAG implementations follow a relatively straightforward workflow:
- Upload documents.
- Extract and clean text.
- Split documents into chunks.
- Generate embeddings.
- Store embeddings in a vector database.
- Retrieve relevant chunks when a user asks a question.
- Pass the retrieved context to an LLM.
This approach works well for many use cases.
However, consider what happens when a document is updated.
Suppose your knowledge base initially contains:
Maintenance Policy — Version 1
Device maintenance must be performed
every 30 days.
A month later, the organization updates the policy.
Maintenance Policy — Version 2
Device maintenance must be performed
every 45 days.
Your ingestion pipeline processes the new document and inserts its embeddings into Qdrant.
But what happens to the previous version?
If the old vectors are not removed, deactivated, or excluded from retrieval, both versions remain searchable.
Now imagine the user asks:
"What is the device maintenance schedule?"
The vector database may return both versions because their semantic similarity scores are high.
The LLM receives two conflicting pieces of information.
Depending on context ordering, prompt instructions, and model behavior, it might select the outdated version, combine both answers, or produce an uncertain response.
This illustrates an important distinction:
Semantic similarity does not guarantee factual validity.
A document can be highly relevant to a question while no longer being authoritative.
For production RAG systems, retrieval must consider more than similarity scores.
It must also consider document ownership, version, approval status, and freshness.
2. Understanding Document Lifecycle in RAG
One common architectural mistake is treating document ingestion as a one-time operation.
In real organizations, knowledge changes continuously.
Policies get updated. Technical manuals are revised. Product specifications evolve. Internal procedures become obsolete.
A reliable RAG system needs to understand the lifecycle of every document.
Instead of storing only the document text and embedding, maintain structured metadata.
For example:
| Field |
Purpose |
| document_id |
Identifies the logical document |
| version |
Identifies the document revision |
| status |
Indicates whether the version is active |
| updated_at |
Records the modification time |
| source_hash |
Detects changes in source content |
| ingestion_status |
Tracks processing progress |
| chunk_index |
Identifies the chunk within the document |
A possible document record might look like this:
{
"document_id": "maintenance-policy",
"version": 2,
"status": "ACTIVE",
"updated_at": "2026-11-12T08:30:00Z",
"source_hash": "sha256:example",
"ingestion_status": "COMPLETED"
}
Each chunk generated from the document should carry the relevant metadata.
For example:
{
"document_id": "maintenance-policy",
"version": 2,
"status": "ACTIVE",
"chunk_index": 3,
"content": "Device maintenance must be performed every 45 days."
}
Now the retrieval system has additional information that can help it distinguish between document versions.
Why source hashes matter
A document's filename is not a reliable way to detect whether its contents have changed.
Someone might upload a modified document using the same filename.
A content hash provides a more dependable mechanism for detecting changes.
For example:
import hashlib
def generate_document_hash(content: bytes) -> str:
return hashlib.sha256(
content
).hexdigest()
If the newly uploaded document has the same hash as the existing version, the ingestion pipeline can avoid unnecessary reprocessing.
However, a hash only identifies content changes. It does not determine whether a document is approved or authoritative.
That decision belongs to the document-management layer.
3. Managing Document Versions in Qdrant
Qdrant is a vector database that supports storing vectors alongside structured payload metadata.
This is useful for RAG systems because retrieval can combine semantic similarity with metadata filtering.
Instead of retrieving every semantically similar chunk, we can restrict retrieval to documents marked as active.
Consider the following implementation:
from qdrant_client.models import (
Filter,
FieldCondition,
MatchValue
)
active_document_filter = Filter(
must=[
FieldCondition(
key="status",
match=MatchValue(
value="ACTIVE"
)
)
]
)
We can apply this filter during retrieval:
results = client.query_points(
collection_name="knowledge_base",
query=embedding,
query_filter=active_document_filter,
limit=5
)
This tells Qdrant to search only among points whose payload contains:
{
"status": "ACTIVE"
}
The important advantage is that the retrieval process now considers document lifecycle metadata in addition to vector similarity.
But there is another problem: activating new versions safely
Imagine your ingestion pipeline performs these operations:
- Deactivates version 1.
- Starts processing version 2.
- Encounters an embedding-generation failure.
- Fails before inserting all the new chunks.
Now the old version is inactive, but the new version is incomplete.
Your knowledge base may no longer have a usable version of that document.
A safer approach is to use staged ingestion.
For example:
VERSION 1: ACTIVE
|
v
UPLOAD VERSION 2
|
v
EXTRACT AND VALIDATE
|
v
GENERATE EMBEDDINGS
|
v
INSERT ALL NEW CHUNKS
|
v
VALIDATE INGESTION
|
v
ACTIVATE VERSION 2
AND RETIRE VERSION 1
The previous version should remain available until the replacement has been successfully processed and validated.
For a single logical document, activation can be coordinated through an authoritative document-version record in PostgreSQL.
The application should determine the active version from this record and use it to construct trusted Qdrant filters.
For example, instead of relying only on a status field:
active_version = get_active_document_version(
document_id="maintenance-policy"
)
The retrieval filter can then include both the document ID and version:
from qdrant_client.models import (
Filter,
FieldCondition,
MatchValue
)
document_filter = Filter(
must=[
FieldCondition(
key="document_id",
match=MatchValue(
value="maintenance-policy"
)
),
FieldCondition(
key="version",
match=MatchValue(value=2)
)
]
)
The version number in this example is illustrative. In an actual application, it should come from the authoritative document registry rather than being hardcoded.
This approach reduces the risk of retrieving chunks from multiple revisions of the same document.
4. Updating Documents Without Breaking Retrieval
A production ingestion pipeline should not simply delete old chunks and insert new ones.
That approach creates unnecessary risk.
Instead, consider a staged document-update workflow.
Step 1: Detect document changes
Calculate the source hash and compare it with the existing document record.
If the content has not changed, avoid unnecessary embedding generation.
Step 2: Create a new document version
Register the new version as PROCESSING.
Do not immediately make it available to normal retrieval requests.
Step 3: Extract and process content
Extract text, clean the content, split it into chunks, and preserve useful metadata such as page numbers, section titles, and source references.
Step 4: Generate embeddings
Generate vectors for the new chunks and store them in Qdrant with the new version metadata.
Step 5: Validate ingestion
Check that:
- All expected chunks were inserted.
- The document contains usable text.
- Required metadata exists.
- Embeddings were generated successfully.
- The document version can be retrieved.
Step 6: Activate the new version
Update the authoritative document registry so that the new version becomes active.
The activation step should be transactional in the metadata database.
If multiple records must change, ensure they are updated in the same transaction.
Step 7: Retire old versions
Once the new version is active and validated, previous versions can be excluded from normal retrieval and eventually deleted according to retention policies.
For large systems, asynchronous cleanup is usually preferable to making the activation process wait for deletion of every old vector.
What if ingestion fails?
The old active version should remain available.
The new version can be marked as FAILED, and the system can retain its error information for debugging and reprocessing.
This makes ingestion recoverable instead of allowing a partial update to silently damage the knowledge base.
5. Preventing Stale Answers During Retrieval
Even with document versioning, there are additional questions worth considering.
How do we know the retrieved content is authoritative?
How do we know the generated answer actually uses the latest approved source?
How can we detect retrieval failures before users report incorrect answers?
There are several useful techniques.
A. Include source metadata in the retrieved context
Do not pass only plain text to the LLM.
Include relevant source information:
context = {
"document_id": "maintenance-policy",
"version": 2,
"updated_at": "2026-11-12T08:30:00Z",
"content": (
"Device maintenance must be "
"performed every 45 days."
)
}
This makes it easier to trace where an answer originated.
B. Preserve citations
When generating an answer, retain the document identifier, version, and chunk reference.
For example:
{
"answer": "Devices should be maintained every 45 days.",
"sources": [
{
"document_id": "maintenance-policy",
"version": 2,
"chunk_index": 3
}
]
}
This improves traceability and makes it easier to investigate incorrect answers.
C. Evaluate retrieval quality
A RAG evaluation pipeline should test more than whether the model generates a fluent response.
Create test questions with known expected document versions.
For example:
| Question |
Expected version |
Expected answer |
| What is the maintenance interval? |
2 |
45 days |
| What is the current device policy? |
2 |
Version 2 |
| What was the previous maintenance interval? |
Historical v1 |
30 days |
The third question is particularly interesting.
Historical information is not necessarily useless. Sometimes users explicitly need to know what a previous policy stated.
This is why simply deleting every old version may not be the right solution.
A better design distinguishes between:
- Current approved knowledge.
- Historical knowledge.
- Draft documents.
- Retired or superseded documents.
The retrieval policy should depend on the user's question and the application's authorization rules.
D. Treat document metadata as trusted application data
Another important security consideration is that retrieved document content is untrusted input.
A document should not be able to declare itself the latest approved version by including text such as "This is now the active policy."
Approval status and version authority must come from the application's trusted metadata layer.
Likewise, document-level access permissions should be enforced during retrieval, not left entirely to the LLM.
6. Production RAG Needs Document Governance
Document versioning solves an important problem, but reliable knowledge retrieval requires a broader architecture.
A production RAG system should have clear responsibilities:
| Component |
Responsibility |
| FastAPI |
Upload, query, and document-management APIs |
| PostgreSQL |
Authoritative document metadata and version registry |
| Qdrant |
Vector storage and metadata-filtered retrieval |
| Embedding service |
Text-to-vector conversion |
| Ingestion worker |
Parsing, chunking, indexing, and validation |
| LLM |
Answer generation using retrieved context |
There are also some operational considerations that developers should not overlook.
Document deletion: If a document is deleted or access is revoked, its vectors and cached representations must be handled appropriately.
Re-indexing: Changes to embedding models or chunking strategies may require rebuilding the vector index.
Observability: Track ingestion failures, document activation events, retrieval versions, and stale-source usage.
Authorization: Ensure users only retrieve documents they are permitted to access. A correct document version is still an incorrect retrieval if the user has no permission to see it.
Cache invalidation: If answers or retrieval results are cached, document updates must invalidate or version those caches. Otherwise, a perfectly updated vector database can still serve an outdated answer through an old cache entry.
These considerations become increasingly important as the knowledge base grows and multiple teams begin managing documents.
Conclusion
A RAG system is only as reliable as the knowledge it retrieves.
Improving embedding quality, changing chunk sizes, or switching to a more powerful LLM will not automatically solve problems caused by outdated documents.
The real challenge is managing knowledge throughout its lifecycle.
A production-ready RAG architecture should understand:
- Which document is authoritative.
- Which version is currently active.
- Whether ingestion completed successfully.
- Whether the retrieved information is still valid.
- Whether the user is authorized to access it.
- Where the generated answer originated.
The most important engineering principle is this:
Retrieval relevance tells you whether information is related to a question. Document governance tells you whether that information should still be trusted.
Building a useful RAG prototype is relatively straightforward.
Building one that consistently retrieves current, authorized, and traceable information requires a much stronger engineering foundation.
And that is where production RAG engineering really begins.
Technologies covered: Python, FastAPI, Qdrant, PostgreSQL, Retrieval-Augmented Generation, LLMs, Vector Search, Document Versioning, Embedding Pipelines.
Suggested C# Corner tags: Artificial Intelligence, Python, FastAPI, Qdrant, RAG, Vector Databases, Software Architecture, Backend Development.
Join the conversation! Your thoughts help the community grow.