Vector search has become an important database capability for applications that need semantic search, recommendation systems, document retrieval, and retrieval-augmented generation (RAG).
For .NET developers using SQL Server, vector support is now available directly through Entity Framework Core. EF Core 10 introduced support for SQL Server's vector data type and VECTOR_DISTANCE(). EF Core 11 extends this with VECTOR_SEARCH() integration and vector indexes.
That creates an interesting engineering question:
How does vector search perform when used through real EF Core workloads?
There is no single answer.
Performance depends on:
This makes vector search a good candidate for controlled benchmarking.
The goal should not be to claim that one approach is universally faster. Instead, developers should measure the trade-off between exactness, latency, resource consumption, and operational complexity.
SQL Server Vector Search in EF Core
SQL Server 2025 introduced the vector data type for storing embeddings and vector operations. EF Core 10 added provider support for this functionality.
A vector property can be represented in EF Core using SqlVector<float>:
public class Article
{
public int Id { get; set; }
public string Title { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
public SqlVector<float> Embedding { get; set; } = null!;
}
The vector column can be configured with a fixed dimension:
protected override void OnModelCreating(
ModelBuilder modelBuilder)
{
modelBuilder.Entity<Article>()
.Property(x => x.Embedding)
.HasColumnType("vector(1536)");
}
The dimension must match the embedding model being used.
A 1536-dimensional embedding and a 3072-dimensional embedding are not interchangeable simply because both use floating-point values.
Exact Vector Search With VECTOR_DISTANCE
The first workload to benchmark is exact nearest-neighbor search.
EF Core translates EF.Functions.VectorDistance() to SQL Server's VECTOR_DISTANCE() function.
For example:
var queryEmbedding =
new SqlVector<float>(queryVector);
var results = await context.Articles
.OrderBy(article =>
EF.Functions.VectorDistance(
"cosine",
article.Embedding,
queryEmbedding))
.Take(10)
.ToListAsync();
Conceptually, SQL Server performs:
Query Vector
|
v
Compare Against Stored Vectors
|
v
Calculate Distance
|
v
Sort
|
v
Return Top K
This is an exact search because the database calculates the distance against the relevant rows rather than using an approximate vector index. Microsoft notes that this can become expensive as the number of vectors grows because the database may need to calculate distances across many rows.
Approximate Vector Search With VECTOR_SEARCH
For larger datasets, SQL Server also provides VECTOR_SEARCH().
EF Core 11 exposes this through VectorSearch():
var results = await context.Articles
.VectorSearch(
article => article.Embedding,
queryEmbedding,
"cosine")
.OrderBy(result => result.Distance)
.Take(10)
.WithApproximate()
.ToListAsync();
The important distinction is:
VECTOR_DISTANCE()
|
v
Exact distance calculation
versus:
VECTOR_SEARCH()
|
v
Vector index
|
v
Approximate nearest neighbors
Microsoft documents WithApproximate() as the switch that causes SQL Server to use the vector index for approximate nearest-neighbor search.
Create a Vector Index
Approximate search requires a vector index.
Configure it in the EF Core model:
protected override void OnModelCreating(
ModelBuilder modelBuilder)
{
modelBuilder.Entity<Article>()
.Property(x => x.Embedding)
.HasColumnType("vector(1536)");
modelBuilder.Entity<Article>()
.HasVectorIndex(
x => x.Embedding,
"cosine");
}
EF Core can generate the corresponding migration for the vector index.
Conceptually, the database receives an index similar to:
CREATE VECTOR INDEX [IX_Articles_Embedding]
ON [Articles] ([Embedding])
WITH (METRIC = COSINE);
The metric must match the intended similarity calculation.
Exact vs Approximate Search
The most important benchmark comparison is:
| Characteristic | Exact Search | Approximate Search |
|---|
| EF Core API | VectorDistance() | VectorSearch() + WithApproximate() |
| Index required | No | Yes |
| Accuracy | Exact nearest neighbors | Approximate |
| Large datasets | Can become expensive | Designed for scalable search |
| Query behavior | Distance calculation | Vector index search |
| Complexity | Lower | Higher |
| Benchmark focus | Accuracy and baseline | Latency and scalability |
Microsoft currently marks VECTOR_SEARCH() and vector indexes as experimental, so applications adopting them should account for API and behavior changes.
Build a Real Benchmark Dataset
A useful benchmark should contain more than a few hundred rows.
Create several dataset sizes, for example:
10,000 vectors
50,000 vectors
100,000 vectors
500,000 vectors
1,000,000 vectors
These are benchmark scenarios, not claims about minimum or maximum supported sizes.
The purpose is to observe how query behavior changes as the dataset grows.
For every dataset, keep the vector dimension constant initially.
Then perform a second experiment with different dimensions.
Benchmark Vector Dimensions
Vector dimensions can affect storage, memory, and computational work.
For example:
384 dimensions
768 dimensions
1536 dimensions
3072 dimensions
The benchmark should use embeddings generated by an appropriate model or deterministic test vectors with the same dimensions.
Do not compare two dimensions while also changing:
Database
Hardware
Metric
Dataset Size
Query Count
Too many variables make the results difficult to interpret.
Generate Test Vectors
For infrastructure benchmarking, deterministic vectors can be useful.
For example:
static float[] CreateVector(
int dimensions,
int seed)
{
var random = new Random(seed);
var vector = new float[dimensions];
for (int i = 0; i < vector.Length; i++)
{
vector[i] =
(float)(random.NextDouble() * 2 - 1);
}
return vector;
}
This allows repeatable tests.
However, deterministic random vectors do not necessarily represent the distribution of embeddings produced by a real model.
For relevance-quality experiments, use actual embeddings.
For infrastructure-only tests, deterministic vectors can make controlled experiments easier.
Benchmark Top-K Values
Do not benchmark only:
TOP 5
Test different result sizes:
Top 1
Top 5
Top 10
Top 25
Top 50
Top 100
The number of requested results can affect execution behavior and resource usage.
For example:
var results = await context.Articles
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(10)
.WithApproximate()
.ToListAsync();
The position of WithApproximate() matters in the current API: Microsoft documents that it should be called after Take().
Benchmark Different Distance Metrics
SQL Server vector indexes currently support metrics including:
Cosine
Euclidean
Dot product
Microsoft documents these as supported vector-index metrics.
A benchmark should therefore avoid assuming that cosine is always the right choice.
Use the metric appropriate for the embedding model and workload.
For example:
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
The metric should remain consistent between index configuration and search.
Benchmark EF Core Query Shape
The database operation is only part of the workload.
EF Core also determines which columns are materialized.
Consider:
var results = await context.Articles
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(10)
.WithApproximate()
.ToListAsync();
Now compare that with a projection:
var results = await context.Articles
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(10)
.WithApproximate()
.Select(x => new
{
x.Value.Id,
x.Value.Title,
x.Distance
})
.ToListAsync();
The second query explicitly requests only the fields required by the application.
This can be important when article content is large.
EF Core 11 Vector Projection Change
There is an additional EF Core 11 consideration.
Starting with EF Core 11, SqlVector<T> properties are no longer loaded by default when entities are materialized. Vector columns can still participate in filtering and ordering, but they are excluded from normal entity projections because embeddings can contain hundreds or thousands of floating-point values.
This is important for benchmarking.
A benchmark comparing EF Core 10 and EF Core 11 should account for this behavior change.
Otherwise, differences in network transfer or materialization may be incorrectly attributed to the vector search algorithm.
Measure Query Execution Time
At minimum, collect:
Elapsed Time
Rows Returned
Errors
For production-oriented testing, add:
p50
p95
p99
Throughput
CPU
Memory
Database IO
Avoid relying only on average latency.
A vector query might produce:
p50 = Low
p95 = Moderate
p99 = High
That can be much more important than the average when the API is serving interactive users.
Measure Database Work Separately
Use SQL Server's diagnostic capabilities to inspect:
Logical Reads
CPU Time
Elapsed Time
Execution Plan
This helps determine whether a slow request comes from:
EF Core
or:
SQL Server
or:
Network
or:
Application Materialization
Without this separation, a benchmark can produce a latency number without explaining why it occurred.
Exact Search Baseline
Start with exact search.
The baseline query is:
var exactResults = await context.Articles
.OrderBy(article =>
EF.Functions.VectorDistance(
"cosine",
article.Embedding,
queryEmbedding))
.Take(10)
.Select(article => new
{
article.Id,
article.Title
})
.ToListAsync();
Record the result.
Then run the equivalent approximate search:
var approximateResults =
await context.Articles
.VectorSearch(
article => article.Embedding,
queryEmbedding,
"cosine")
.OrderBy(result => result.Distance)
.Take(10)
.WithApproximate()
.Select(result => new
{
result.Value.Id,
result.Value.Title,
result.Distance
})
.ToListAsync();
Now you have two measurements:
Exact
vs
Approximate
Measure Recall, Not Just Latency
An approximate search benchmark should not only ask:
How fast is it?
It should also ask:
How often does it return the same relevant neighbors as exact search?
For example:
Exact Top 10
|
v
Ground Truth
Approximate Top 10
|
v
Compare
Calculate a metric such as recall@10:
Recall@10 =
Relevant Results Retrieved
--------------------------
Relevant Results Expected
The exact formula should match the evaluation methodology.
The important principle is that approximate search involves a performance-versus-result-quality trade-off.
Example Result Table
A benchmark report can use:
| Dataset | Search | Top K | p50 | p95 | RPS | Recall@K |
|---|
| 50K | Exact | 10 | Measure | Measure | Measure | 100% baseline |
| 50K | Approximate | 10 | Measure | Measure | Measure | Measure |
| 500K | Exact | 10 | Measure | Measure | Measure | 100% baseline |
| 500K | Approximate | 10 | Measure | Measure | Measure | Measure |
| 1M | Exact | 10 | Measure | Measure | Measure | 100% baseline |
| 1M | Approximate | 10 | Measure | Measure | Measure | Measure |
Do not populate this table with assumed numbers.
The exact results are dependent on the dataset, hardware, vector distribution, SQL Server configuration, and index configuration.
Benchmark Cold and Warm Queries
Database benchmarks should distinguish between:
Cold
and:
Warm
execution.
A cold query may involve:
Disk IO
Cache Population
Plan Compilation
A warm query may benefit from:
Buffer Cache
Existing Query Plan
Already-Loaded Data
Run both where the distinction matters.
Benchmark Concurrent Searches
A single query does not represent a production RAG system.
Run concurrent searches:
1 request
10 requests
50 requests
100 requests
Measure:
Throughput
p95
p99
CPU
Memory
Database waits
Connection pool behavior
The database may behave very differently under concurrent vector searches.
Avoid Creating One DbContext Per Vector Comparison
Use normal EF Core lifetime management.
For ASP.NET Core applications, a scoped DbContext is commonly used:
builder.Services.AddDbContext<AppDbContext>(
options =>
options.UseSqlServer(connectionString));
Do not create an unrealistic benchmark architecture simply to produce a faster number.
The benchmark should resemble the production application's database access pattern.
Use AsNoTracking for Read-Only Search
Vector search is often read-only.
For result queries that do not need change tracking:
var results = await context.Articles
.AsNoTracking()
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(10)
.WithApproximate()
.Select(x => new
{
x.Value.Id,
x.Value.Title,
x.Distance
})
.ToListAsync();
This avoids unnecessary EF Core change-tracking work.
Benchmark with and without AsNoTracking() if you want to understand its contribution in your application.
Do Not Load Embeddings Unnecessarily
Embeddings can be large.
Suppose:
1536 dimensions
are stored as single-precision floating-point values.
The vector contains many values that the application often does not need to send back to the API client.
EF Core 11 addresses this by not loading SqlVector<T> properties by default during entity materialization.
Explicit projections are still a good practice:
.Select(x => new
{
x.Value.Id,
x.Value.Title,
x.Distance
})
Return only what the application needs.
Benchmark Hybrid Search
Vector search is not necessarily a replacement for traditional text search.
SQL Server also provides full-text search capabilities.
EF Core supports SQL Server full-text search functions, including table-valued functions that expose ranking information.
A hybrid search can combine:
Keyword Search
+
Vector Search
The resulting architecture is:
User Query
|
+----------+
| |
v v
Full Text Vector Search
| |
+----+-----+
|
v
Rank Fusion
|
v
Top Results
Reciprocal Rank Fusion
One documented approach is Reciprocal Rank Fusion, or RRF.
Conceptually:
RRF Score =
1 / (k + TextRank)
+
1 / (k + VectorRank)
EF Core's current SQL Server vector-search documentation demonstrates combining full-text and vector results through a full join and calculating an RRF score.
A simplified LINQ pattern is:
var results = await context.Articles
.FreeTextTable<Article, int>(
textualQuery,
topN: 20)
.Join(
context.Articles,
fts => fts.Key,
article => article.Id,
(fts, article) => new
{
Article = article,
fts.Rank
})
.FullJoin(
context.Articles
.VectorSearch(
article => article.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(20)
.WithApproximate(),
fts => fts.Article.Id,
vector => vector.Value.Id,
(fts, vector) => new
{
Article =
fts != null
? fts.Article
: vector.Value,
FullTextRank =
fts == null
? null
: (int?)fts.Rank,
VectorDistance =
vector == null
? null
: (double?)vector.Distance
})
.ToListAsync();
The exact production query should be optimized and validated against the actual SQL generated by EF Core.
Why Hybrid Search Matters
Keyword search is strong when the user enters:
"EF Core 11"
while semantic search may be better for:
"How can I prevent large embeddings from being returned?"
A hybrid system can combine both signals.
However, hybrid search also introduces additional query work.
Therefore, benchmark:
Vector Only
Full Text Only
Hybrid
rather than assuming hybrid search is always worth its additional complexity.
Benchmark SQL Generation
Do not assume the LINQ query translates exactly as expected.
Inspect generated SQL:
var query = context.Articles
.VectorSearch(
x => x.Embedding,
queryEmbedding,
"cosine")
.OrderBy(x => x.Distance)
.Take(10)
.WithApproximate();
Console.WriteLine(
query.ToQueryString());
The generated SQL can reveal whether the expected SQL Server vector functionality is being used.
Microsoft documents that the approximate EF Core query translates to SQL Server's VECTOR_SEARCH() table-valued function and WITH APPROXIMATE syntax.
Benchmark the Database Directly
EF Core should not be the only benchmark.
Run the equivalent SQL directly against SQL Server.
This creates:
Benchmark A
SQL Server Direct
Benchmark B
EF Core
Difference
=
ORM + Application Overhead
If SQL Server takes 20 ms but the complete API takes 80 ms, vector search itself is probably not responsible for the entire latency.
This is one of the most useful diagnostic comparisons.
Common Mistakes
Comparing Different Embedding Dimensions
Keep dimensions consistent when comparing search strategies.
Measuring Only Average Latency
Include p95 and p99.
Ignoring Recall
Approximate search is a quality-performance trade-off.
Loading Complete Entities
Project only required fields.
Ignoring Generated SQL
Always inspect the SQL for important benchmark queries.
Benchmarking Only One Dataset Size
Scalability is the primary reason to compare exact and approximate approaches.
Ignoring Concurrent Requests
Single-user performance does not represent production load.
Mixing Embedding Generation Into Database Benchmarks
Embedding generation is a separate workload.
Measure:
Embedding Generation
and:
Vector Search
independently.
Treating Experimental APIs as Stable
Microsoft currently marks VECTOR_SEARCH() and vector indexes as experimental.
Returning Embeddings to the API
Applications generally need the search results, not the full vector data.
Troubleshooting
The Vector Column Does Not Exist
Check that the migration created the correct column type:
vector(1536)
and that the dimension matches the generated embeddings.
Vector Search Throws a Dimension Error
Verify:
Embedding Model
Vector Dimension
Database Column Dimension
Query Vector Dimension
All must be compatible.
Approximate Search Is Not Using the Vector Index
Verify:
Vector Index Exists
Correct Metric
Take() Present
WithApproximate() Present
Microsoft documents that WithApproximate() is required to request approximate nearest-neighbor behavior and that it should follow Take().
Results Are Slower Than Expected
Inspect:
Execution Plan
Logical Reads
CPU
Dataset Size
Vector Index
Concurrency
Do not assume the ORM is responsible.
EF Core Returns Unexpectedly Large Results
Use projection:
.Select(x => new
{
x.Value.Id,
x.Value.Title,
x.Distance
})
This also makes the intended API payload explicit.
Exact Search Is Acceptable
That may be the correct architecture.
SQL Server's documentation recommends exact search for smaller vector sets, with a general recommendation around fewer than 50,000 vectors, while larger datasets may benefit from approximate search. The actual threshold should still be validated against the application's workload.
Production Benchmark Matrix
A serious evaluation can use:
| Dimension | Values |
|---|
| Dataset | 10K / 50K / 500K / 1M |
| Dimensions | Model-dependent |
| Search | Exact / Approximate |
| Top K | 1 / 5 / 10 / 50 |
| Metric | Cosine / Euclidean / Dot |
| Concurrency | 1 / 10 / 50 / 100 |
| Query state | Cold / Warm |
| API | Direct DB / EF Core |
| Search mode | Vector / Full Text / Hybrid |
| Metrics | p50 / p95 / p99 / RPS / CPU / IO |
This produces a benchmark that can actually guide architecture decisions.
Recommended Architecture
A practical RAG architecture using SQL Server and EF Core can look like:
User Question
|
v
Embedding Generator
|
v
Query Embedding
|
v
EF Core
|
v
SQL Server Vector Search
|
+---- Vector Results
|
+---- Optional Full-Text Results
|
v
Rank / Filter
|
v
Top Documents
|
v
LLM
|
v
Response
The database remains responsible for retrieval.
The application remains responsible for orchestration.
The model remains responsible for generating the final response.
Best Practices
Benchmark exact and approximate search separately.
Use the same embedding dimensions when comparing algorithms.
Measure recall alongside latency for approximate search.
Test multiple dataset sizes.
Test multiple Top-K values.
Measure p50, p95, and p99 latency.
Test concurrent requests.
Inspect generated SQL.
Compare EF Core execution with direct SQL.
Use projections instead of loading unnecessary columns.
Use AsNoTracking() for read-only workloads where appropriate.
Do not regenerate embeddings during a database benchmark.
Benchmark cold and warm database behavior separately.
Evaluate vector-only and hybrid search independently.
Keep vector-index metrics aligned with the intended similarity calculation.
Treat current VECTOR_SEARCH() and vector-index APIs as experimental.
Record exact .NET, EF Core, SQL Server, OS, and hardware versions.
Do not publish benchmark numbers without the dataset and methodology.
Frequently Asked Questions
Does EF Core support SQL Server vector search?
Yes. EF Core 10 introduced support for SQL Server's vector type and VECTOR_DISTANCE(). EF Core 11 adds VectorSearch() and vector-index configuration.
What is the difference between VECTOR_DISTANCE and VECTOR_SEARCH?
VECTOR_DISTANCE() calculates the distance between vectors and can be used for exact nearest-neighbor queries.
VECTOR_SEARCH() searches a collection of vectors and can use a vector index for approximate nearest-neighbor search when WithApproximate() is specified.
Is approximate vector search always faster?
It is designed to improve performance for large datasets, but the actual result depends on the data, index configuration, hardware, concurrency, and query shape. Benchmark it rather than assuming a fixed improvement.
Does approximate search return the same results as exact search?
Not necessarily.
Approximate nearest-neighbor search trades some exactness for search efficiency. That is why recall should be measured alongside latency.
What vector metrics does SQL Server support?
The current EF Core documentation lists cosine, Euclidean, and dot-product metrics for vector indexes.
Should embeddings be returned from EF Core queries?
Usually not.
Applications generally need the matching document and similarity information rather than the complete embedding. EF Core 11 also changed vector-property loading so vectors are not included in normal entity materialization by default.
Can SQL Server vector search be combined with full-text search?
Yes.
EF Core's SQL Server provider supports combining full-text search and vector search, including a Reciprocal Rank Fusion approach for combining rankings.
Is VECTOR_SEARCH production-ready?
The current Microsoft documentation marks VECTOR_SEARCH() and vector indexes as experimental and subject to change. Teams should therefore validate the specific EF Core and SQL Server versions used in production.
How large should the vector dataset be before using approximate search?
There is no universal cutoff.
Microsoft's SQL Server documentation gives a general recommendation to use exact search when there are fewer than about 50,000 vectors, but the actual decision should be based on measured latency, resource usage, and relevance requirements.
Conclusion
SQL Server's native vector capabilities give .NET developers an opportunity to keep structured data, application data, and semantic search within the same database platform.
EF Core makes that capability accessible through LINQ:
C#
|
v
EF Core
|
v
SQL Server
|
+---- VECTOR_DISTANCE()
|
+---- VECTOR_SEARCH()
|
+---- Vector Index
The most important benchmarking distinction is between exact and approximate search.
Exact search provides a useful ground-truth baseline:
Query Vector
|
v
Compare Candidates
|
v
Exact Top K
Approximate search introduces an index-assisted retrieval path:
Query Vector
|
v
Vector Index
|
v
Approximate Top K
The correct choice depends on the workload.
A meaningful benchmark should therefore measure:
Dataset Size
+
Vector Dimensions
+
Top K
+
Search Metric
+
Concurrency
+
Latency
+
Throughput
+
CPU
+
IO
+
Recall
It should also separate EF Core overhead from SQL Server execution and separate vector-search latency from embedding-generation and LLM latency.
The most important lesson is:
Do not choose exact or approximate vector search based on a generic performance claim. Establish exact search as the relevance baseline, measure approximate search against it, and choose the architecture based on the latency, scalability, resource consumption, and retrieval quality your application actually requires.
Join the conversation! Your thoughts help the community grow.