Introduction
One of the biggest limitations of AI agents is that they do not naturally remember previous interactions. While Large Language Models (LLMs) can process context provided in a prompt, they typically lose that context once the conversation ends.
Imagine a customer support agent that forgets every previous conversation or a personal assistant that cannot remember user preferences. Such systems quickly become frustrating and ineffective.
This is where a memory layer becomes essential.
A memory layer allows AI agents to store, retrieve, and use information from previous interactions. Combined with semantic search, agents can recall relevant information even when the exact words used in a query differ from the original conversation.
In this article, you'll learn how to build a memory layer for AI agents using PostgreSQL and semantic search, understand the architecture involved, and explore best practices for building scalable memory systems in .NET applications.
Why AI Agents Need Memory
Without memory, an AI agent treats every interaction as a new conversation.
Example:
User:
My favorite programming language is C#.
Later:
User:
What programming language do I prefer?
Without memory:
I don't know.
With memory:
Your preferred programming language is C#.
Memory enables:
Types of AI Agent Memory
AI systems often use multiple memory types.
Short-Term Memory
Stores information for the current session.
Examples:
Current conversation
Active workflow
Temporary context
Current chat session
Long-Term Memory
Stores information across sessions.
Examples:
User preferences
Historical conversations
Business knowledge
Stored user profile
Semantic Memory
Stores information that can be retrieved based on meaning rather than exact keywords.
Example:
Stored memory:
User likes ASP.NET Core development.
Query:
What technologies does the user enjoy working with?
Semantic search can identify the relationship between the query and the stored memory.
What Is Semantic Search?
Traditional search relies on exact keywords.
Example:
Stored content:
ASP.NET Core is used for building web applications.
Search:
Web development framework
Keyword search may fail.
Semantic search works differently.
It converts text into vector embeddings and compares meaning rather than words.
Example:
ASP.NET Core
and
Web development framework
may be recognized as closely related concepts.
This makes semantic search ideal for AI memory systems.
Why PostgreSQL?
PostgreSQL has become a popular choice for AI applications because it offers:
Reliability
Scalability
ACID compliance
Advanced indexing
Open-source flexibility
Vector search support
With vector extensions, PostgreSQL can store embeddings alongside traditional relational data.
This allows developers to combine structured and semantic search capabilities in a single database.
Memory Layer Architecture
A typical architecture might look like this:
User Request
|
v
AI Agent
|
v
Memory Service
|
v
PostgreSQL
|
v
Semantic Search
|
v
Relevant Memories
Workflow:
User submits a request.
Agent queries memory.
Relevant memories are retrieved.
Context is added to the prompt.
Agent generates a response.
Creating the Memory Model
Let's create a simple memory entity.
public class MemoryRecord
{
public Guid Id { get; set; }
public string UserId { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
public DateTime CreatedAt { get; set; }
}
This model stores information that can later be retrieved by the agent.
Creating a Memory Service
The memory service handles storage and retrieval.
public interface IMemoryService
{
Task SaveAsync(
string userId,
string content);
Task<List<MemoryRecord>> SearchAsync(
string query);
}
This abstraction keeps the implementation flexible.
Saving Memories
Example implementation:
public async Task SaveAsync(
string userId,
string content)
{
var memory = new MemoryRecord
{
Id = Guid.NewGuid(),
UserId = userId,
Content = content,
CreatedAt = DateTime.UtcNow
};
_dbContext.Memories.Add(memory);
await _dbContext.SaveChangesAsync();
}
Every important interaction can be stored for future use.
Retrieving Memories
Basic retrieval:
public async Task<List<MemoryRecord>>
SearchAsync(string query)
{
return await _dbContext.Memories
.Where(m => m.Content.Contains(query))
.ToListAsync();
}
This works for keyword searches but does not provide semantic understanding.
For production systems, vector search is preferred.
Understanding Embeddings
Embeddings are numerical representations of text.
Example:
C# Development
might become:
[0.125, 0.348, 0.782, ...]
Similarly:
Building .NET Applications
might generate a nearby vector.
Because the vectors are similar, semantic search can identify the relationship.
Storing Embeddings
Extend the memory model.
public class MemoryRecord
{
public Guid Id { get; set; }
public string UserId { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
public float[] Embedding { get; set; } = Array.Empty<float>();
}
Each memory now stores both text and its vector representation.
Semantic Search Workflow
The process typically follows these steps:
Step 1: User Query
What technologies does the user prefer?
Step 2: Generate Query Embedding
Convert the query into a vector.
Step 3: Similarity Search
Search PostgreSQL for similar vectors.
Step 4: Retrieve Relevant Memories
Example:
User enjoys ASP.NET Core development.
Step 5: Inject Context
Add retrieved memories to the prompt.
Step 6: Generate Response
The AI produces a personalized answer.
Using Memory in AI Agents
Consider this interaction.
First conversation:
User:
I prefer PostgreSQL over SQL Server.
Memory stored:
User prefers PostgreSQL over SQL Server.
Later:
User:
Which database should I use?
The agent retrieves the memory and generates:
Based on your previous preference,
PostgreSQL may be a good choice.
The experience becomes significantly more personalized.
Real-World Use Cases
Memory layers are useful in many AI applications.
Customer Support
Store:
Previous tickets
Customer preferences
Historical interactions
AI Assistants
Remember:
User interests
Frequently used commands
Personal preferences
Knowledge Management
Store:
Organizational knowledge
Team decisions
Project information
Development Assistants
Remember:
Performance Considerations
As memory grows, performance becomes important.
Archive Old Memories
Move outdated records to cold storage.
Limit Retrieved Context
Only return the most relevant memories.
Use Indexes
Proper indexing improves search performance.
Monitor Query Latency
Track retrieval performance regularly.
These practices help maintain scalability.
Security Considerations
Memory often contains sensitive information.
Encrypt Sensitive Data
Protect stored information using encryption.
Implement Access Controls
Users should only access their own memories.
Audit Memory Access
Track who accessed what information.
Define Retention Policies
Remove unnecessary data after a defined period.
Security should be treated as a core requirement.
Best Practices
Store Important Information Only
Avoid saving every interaction.
Focus on information with long-term value.
Use Semantic Search
Keyword matching alone is often insufficient.
Keep Memory Relevant
Remove outdated or irrelevant records.
Separate User Data
Ensure strong isolation between users.
Monitor Storage Growth
Memory systems can grow rapidly over time.
Regular monitoring prevents performance issues.
Common Challenges
Memory Overload
Too much context can reduce response quality.
Duplicate Information
Repeated memories create noise.
Retrieval Accuracy
Poor search quality leads to poor responses.
Storage Costs
Long-term memory requires additional infrastructure.
Thoughtful architecture helps address these challenges.
Conclusion
Memory is one of the most important capabilities for building intelligent AI agents. Without it, agents cannot maintain context, personalize responses, or learn from previous interactions. By combining PostgreSQL with semantic search techniques, developers can create memory systems that allow AI agents to retrieve relevant information based on meaning rather than exact keywords.
For .NET developers, PostgreSQL provides a reliable and scalable foundation for implementing long-term memory while supporting both structured data and vector-based search. When combined with embeddings, semantic retrieval, and strong security practices, memory layers can significantly improve the quality and usefulness of AI-powered applications.
Whether you're building customer support systems, personal assistants, enterprise knowledge platforms, or autonomous agents, a well-designed memory layer is a critical step toward creating more capable and context-aware AI experiences.
Join the conversation! Your thoughts help the community grow.