Azure Cosmos DB AI Query Assistance: Measuring Accuracy and RU Efficiency

Writing database queries becomes more difficult as applications grow. Developers need to understand the data model, partitioning strategy, available indexes, filtering requirements, and the application's expected result shape.

AI-assisted query generation can reduce some of that work. A developer can describe a requirement in natural language and ask an AI tool to generate a query for Azure Cosmos DB.

That can be useful, but a generated query should not be considered production-ready simply because it executes successfully.

For Azure Cosmos DB for NoSQL, a useful evaluation needs to consider at least two dimensions:

Accuracy: Does the query return the correct data?
RU efficiency: Does it perform the operation without unnecessary request-unit consumption?

Latency, partition behavior, result size, and query safety are also valuable measurements.

This makes AI query assistance a good candidate for a controlled engineering experiment rather than a simple feature demonstration.

Why Query Accuracy Is Only the First Test

Consider a requirement such as:

Find active orders for customer 1001
created during the last 30 days.

An AI assistant might generate:

SELECT c.id, c.status, c.total
FROM c
WHERE c.customerId = @customerId
AND c.status = @status
AND c.createdAt >= @startDate

The query looks reasonable.

But several questions remain:

Are all required records returned?
Are inactive orders excluded?
Is the date comparison correct?
Is the partition key being used?
How many RUs does the query consume?
Is the application retrieving more fields than necessary?
Does the query behave similarly as data volume grows?

A good evaluation therefore looks like:

Natural-Language Requirement
             |
             v
       AI Query Assistant
             |
             v
       Generated Query
             |
       +-----+-----+------+
       |           |      |
       v           v      v
  Correctness     RU    Latency
       |           |      |
       +-----+-----+------+
             |
             v
       Engineering Result

Understanding Request Units

Azure Cosmos DB uses Request Units (RUs) to represent the computational cost of database operations.

For AI-generated queries, RU consumption is particularly interesting because the assistant may produce several logically equivalent queries with different execution characteristics.

For example:

Query	Result	RU Cost	Interpretation
A	Correct	Measured	Baseline
B	Correct	Measured	Compare
C	Incorrect	Measured	Reject
D	Correct	High	Investigate

The values must come from an actual experiment. RU consumption varies according to factors such as data size, indexing, partitioning, query shape, and returned data.

Therefore, there is no single RU value that should be presented as the expected cost of a query pattern.

Build a Trusted Baseline

The first step is to create a query written or reviewed by an experienced developer.

Suppose the container uses:

/customerId

as its partition key.

A baseline query could be:

SELECT c.id, c.status, c.total
FROM c
WHERE c.customerId = @customerId
AND c.status = @status
AND c.createdAt >= @startDate

This query becomes the reference point for evaluating the AI-generated version.

The baseline should be reviewed for:

Correctness
Partition strategy
Required projection
Parameterization
Expected workload

The goal is not to prove that the human query is perfect. It is to establish a controlled reference.

Create Representative Test Data

AI query evaluation is only meaningful when the dataset represents the application's data model.

A sample document might look like:

{
  "id": "order-10001",
  "customerId": "customer-1001",
  "status": "Active",
  "total": 249.50,
  "createdAt": "2026-08-20T10:30:00Z",
  "items": [
    {
      "productId": "product-10",
      "quantity": 2
    }
  ]
}

A useful dataset should include:

Multiple customers
Different order statuses
Different dates
Multiple products
Empty and non-empty arrays
Different document sizes
Realistic distribution of records

It should also contain enough data to make partitioning and query behavior meaningful.

A query that looks efficient against 100 documents may not behave the same way against a much larger dataset.

Give the AI Assistant Schema Context

AI query generation improves when the assistant has accurate schema information.

For example:

{
  "container": "orders",
  "partitionKey": "/customerId",
  "fields": {
    "id": "string",
    "customerId": "string",
    "status": "string",
    "total": "number",
    "createdAt": "datetime"
  }
}

The assistant should also understand important constraints:

Use parameterized queries.
Partition key: customerId.
Return only required fields.
Do not generate write operations.

Without this information, the model may infer field names or relationships incorrectly.

Test Query Accuracy

Accuracy should be evaluated against a known expected result.

For example:

var expectedIds = new HashSet<string>
{
    "order-10001",
    "order-10004",
    "order-10009"
};

After executing the AI-generated query:

var actualIds = results
    .Select(x => x.Id)
    .ToHashSet();

Assert.Equal(expectedIds, actualIds);

For more complex queries, compare the complete expected result rather than only checking that some records exist.

This helps detect:

Missing records
Extra records
Incorrect filtering
Incorrect sorting
Incorrect projections

Measure RU Consumption in C#

The Cosmos DB SDK exposes request-charge information through the query response.

A simple measurement can be implemented like this:

var iterator = container.GetItemQueryIterator<Order>(
    queryDefinition);

double totalRequestCharge = 0;

while (iterator.HasMoreResults)
{
    var response = await iterator.ReadNextAsync();

    totalRequestCharge += response.RequestCharge;

    foreach (var order in response)
    {
        Console.WriteLine(order.Id);
    }
}

Console.WriteLine(
    $"Total request charge: {totalRequestCharge} RU");

For a multi-page query, capture the total request charge across all responses.

Comparing only the first page can produce an incomplete measurement.

Measure Latency Separately

RU cost and latency are related but not identical.

Measure query execution time separately:

var stopwatch = Stopwatch.StartNew();

var results = await ExecuteQueryAsync();

stopwatch.Stop();

Console.WriteLine(
    $"Elapsed: {stopwatch.ElapsedMilliseconds} ms");

For a meaningful benchmark, execute the same workload multiple times and record appropriate statistics rather than relying on one execution.

Useful measurements include:

Average
Median
P95
P99

The exact metrics should depend on the workload being tested.

Test Partition-Aware Queries

Partitioning is one of the most important parts of Cosmos DB query design.

Suppose:

/customerId

is the partition key.

A query that supplies a specific customer ID can provide the database with useful partition information:

SELECT c.id, c.total
FROM c
WHERE c.customerId = @customerId
AND c.status = @status

An AI assistant that ignores the partition model may generate a query such as:

SELECT c.id, c.total
FROM c
WHERE c.status = @status

That may still be functionally correct, depending on the dataset.

However, it may have a very different cost profile.

This is exactly why correctness and RU efficiency need to be measured separately.

Projection Matters

AI-generated queries may retrieve entire documents when only a few fields are needed.

For example:

SELECT *
FROM c
WHERE c.status = @status

If the application only needs an ID and total:

SELECT c.id, c.total
FROM c
WHERE c.status = @status

The second query expresses a more focused data requirement.

When evaluating AI query generation, track whether the assistant consistently selects only the fields required by the task.

This is both a performance and data-minimization concern.

Test Different Natural-Language Prompts

Do not evaluate the assistant using one carefully written prompt.

Create multiple phrasings of the same requirement.

For example:

Find active orders for customer 1001.

Show orders that are currently active
for customer 1001.

List customer 1001's active orders.

Return active orders belonging to customer 1001.

The expected semantic result is the same.

If materially different queries are generated, compare their correctness and cost.

This helps determine whether the assistant is robust to normal variations in developer language.

Build a Query Evaluation Matrix

A practical test suite can look like this:

Scenario	Expected Result	AI Correct?	RU	Latency	Partition-Aware
Customer lookup	Known	Measure	Measure	Measure	Measure
Status filter	Known	Measure	Measure	Measure	Measure
Date range	Known	Measure	Measure	Measure	Measure
Customer + status	Known	Measure	Measure	Measure	Measure
Projection	Known	Measure	Measure	Measure	Measure
Sorting	Known	Measure	Measure	Measure	Measure
Nested data	Known	Measure	Measure	Measure	Measure

This produces much more useful evidence than a few manually tested examples.

Compare AI Queries With Expert-Written Queries

One of the strongest experiments is direct comparison.

Natural-Language Requirement
          |
     +----+----+
     |         |
     v         v
Expert SQL   AI SQL
     |         |
     +----+----+
          |
          v
       Cosmos DB
          |
     +----+----+
     |    |    |
     v    v    v
 Correct RU  Latency

For every query, record:

Result correctness
RU consumption
Latency
Number of records returned
Projection
Partition behavior

This gives the team a measurable basis for deciding whether AI assistance is appropriate for a particular workload.

Test Query Stability

An AI assistant may generate different queries for the same requirement.

For example, run the same prompt multiple times where the evaluation setup permits it.

Record:

Prompt
  |
  +-- Run 1 -> Query A
  |
  +-- Run 2 -> Query B
  |
  +-- Run 3 -> Query C

Then compare the results.

If all queries are correct but have significantly different RU costs, the system may need stronger query-generation instructions or a validation layer.

Add Query Validation

Generated queries should pass through validation before execution.

A conceptual interface might be:

public interface IQueryValidator
{
    Task<QueryValidationResult> ValidateAsync(
        string query);
}

The validator can check:

Allowed operation
Allowed container
Required parameters
Maximum result size
Required partition information
Restricted fields

The AI should not be the final authority on whether a query is safe to execute.

Protect Against Write Operations

If the assistant is intended only for query assistance, keep the capability read-only.

A tool contract can make that explicit:

Allowed:
SELECT

Not allowed:
INSERT
UPDATE
DELETE

The database identity should reinforce this restriction.

Defense in depth is important because an application-level restriction should not be the only control.

Cost Controls for AI Query Assistance

An agent can generate multiple queries during one task.

That creates a different cost model from a human writing one query manually.

A production system can monitor cumulative RU usage:

Agent Task
   |
   +-- Query 1 -> 4 RU
   +-- Query 2 -> 8 RU
   +-- Query 3 -> 12 RU
   +-- Query 4 -> 5 RU
   |
   v
Total = 29 RU

A task-level budget can then be enforced.

For example:

Within budget
     |
     v
Continue

Budget exceeded
     |
     v
Stop / Require Approval

The appropriate threshold depends entirely on the application and workload.

Common Mistakes

Measuring Only Whether the Query Executes

Execution success does not establish correctness.

Measuring Only RU Cost

A cheap query that returns incorrect data is still a failed query.

Using Tiny Datasets

Small datasets can hide partition and scalability problems.

Giving the Assistant Incomplete Schema Information

The model may invent fields or misunderstand relationships.

Ignoring Result Size

Returning thousands of records to an AI assistant can increase cost and context usage.

Comparing Different Environments

Keep the database, dataset, indexing configuration, and workload consistent between baseline and AI tests.

Treating AI Output as Trusted SQL

Generated queries should be validated before execution.

Troubleshooting

AI Query Returns Extra Records

Compare the generated filtering conditions with the natural-language requirement.

Check:

Status filter
Date range
Tenant/customer condition
Case sensitivity
Null behavior

RU Consumption Is Much Higher

Inspect:

Partition-key usage
Projection
Number of records returned
Filtering strategy
Sorting
Dataset size

Then compare the execution behavior with the expert baseline.

AI Uses the Wrong Field

Improve schema context and explicitly identify important fields.

For example:

Customer identifier:
customerId

Do not use:
customer
customerID
customer.id

This reduces ambiguity.

Same Prompt Produces Different Query Shapes

Use stronger query-generation constraints and validate the generated query before execution.

Best Practices

Create a trusted baseline query for every important workload.
Use representative test data.
Provide accurate schema and partition-key information.
Evaluate correctness separately from efficiency.
Measure RU consumption across complete query execution.
Measure latency separately.
Test multiple natural-language formulations.
Compare AI-generated queries with expert-written queries.
Restrict the assistant to read-only access where appropriate.
Validate generated queries before execution.
Limit result size.
Track cumulative RU consumption for multi-query agent tasks.
Record generated queries for later analysis.
Test behavior at realistic data volumes.

Advantages and Disadvantages

Advantages

Reduces the effort required to write simple database queries.
Allows developers to describe data requirements naturally.
Can help developers explore unfamiliar datasets.
Provides an opportunity to automate query generation.
Query quality can be measured objectively through correctness and RU consumption.

Disadvantages

AI-generated queries can be logically incorrect.
Correct queries may still be inefficient.
Query generation can vary between prompts or executions.
Incomplete schema context can produce invalid queries.
Uncontrolled agents can generate unnecessary database costs.
Database access introduces security and data-governance concerns.

A Practical Production Architecture

A production implementation should avoid allowing an AI assistant to send unrestricted query text directly to Cosmos DB.

A safer design is:

Developer
    |
    v
AI Query Assistant
    |
    v
Schema Context
    |
    v
Query Generator
    |
    v
Query Validator
    |
    +---- Reject ----> Audit
    |
    v
Read-Only Cosmos Identity
    |
    v
Azure Cosmos DB
    |
    +----> Results
    +----> RU Charge
    |
    v
Evaluation / Application

This design separates generation from execution.

The AI proposes a query.

The application decides whether the query is permitted.

The database identity provides another authorization boundary.

Conclusion

Azure Cosmos DB AI query assistance should be evaluated as an engineering capability, not simply as a convenient way to generate SQL-like queries.

The strongest evaluation measures both accuracy and RU efficiency. A query that returns exactly the expected data but consumes significantly more resources than a carefully written alternative may not be suitable for a high-volume production workload. Likewise, a low-cost query that returns incomplete or incorrect results is not useful.

A repeatable benchmark should use realistic data, a trusted expert baseline, accurate schema context, and controlled execution. Record correctness, RU consumption, latency, result size, and partition behavior for each generated query.

The key lesson is simple: AI-generated queries should be treated as candidates for execution, not as automatically optimized database code. A validation layer, appropriate identity permissions, and measurable cost controls turn query assistance from an interesting AI feature into a more reliable engineering capability.