Azure Cosmos DB AI Query Assistance: Measuring Accuracy and RU Efficiency

Introduction

AI-assisted database querying is becoming useful for developers who work with large and unfamiliar data models. Instead of manually writing every query, a developer can describe the required information in natural language and let an AI system help construct the database query.

For Azure Cosmos DB applications, this can be especially useful when developers need to explore documents, filter records, or build queries against containers they did not design themselves.

However, a generated query should not be considered successful simply because it executes without an error.

A useful query must satisfy at least two requirements:

It should return the correct data.
It should use database resources efficiently.

For Azure Cosmos DB, Request Units (RUs) provide an important measure of database consumption. Two queries can return the same records while consuming different amounts of RUs.

This makes AI-assisted query generation an interesting engineering problem:

Can AI-generated Cosmos DB queries achieve the same correctness as developer-written queries while maintaining reasonable RU efficiency?

The answer should come from measurement rather than assumption.

What Is AI Query Assistance?

AI query assistance allows a developer or application to describe a data requirement in natural language.

For example:

Find the five most recent completed orders
for customer 1001.

An AI system may generate a query similar to:

SELECT TOP 5
    c.id,
    c.customerId,
    c.status,
    c.total,
    c.createdAt
FROM c
WHERE c.customerId = @customerId
  AND c.status = @status
ORDER BY c.createdAt DESC

The application can then validate and execute the query.

The important distinction is:

Natural Language
       |
       v
AI Query Generation
       |
       v
Validation
       |
       v
Cosmos DB
       |
       +--> Result
       +--> RU Charge

The AI generates the candidate query, but the application should remain responsible for validation and execution policy.

Why Accuracy and RU Efficiency Must Be Measured Together

Consider two queries that return exactly the same five orders.

Query A
Correct result
Lower RU consumption

Query B
Correct result
Higher RU consumption

Both are functionally correct.

However, Query A may be preferable for a frequently executed production workload.

Now consider:

Query C
Lower RU consumption
Incorrect result

Query C is not a successful optimization.

Therefore, the benchmark should use a two-stage evaluation:

Stage 1
Correctness

Stage 2
RU Efficiency

Never optimize a query before confirming that it answers the business requirement correctly.

Define a Representative Cosmos DB Dataset

Start with a realistic document model.

For example:

{
  "id": "order-1001",
  "customerId": "customer-42",
  "status": "Completed",
  "total": 749.50,
  "createdAt": "2026-08-20T10:15:00Z",
  "items": [
    {
      "productId": "p-100",
      "quantity": 2
    }
  ]
}

The benchmark should contain enough data to exercise realistic query behavior.

Consider multiple dataset sizes:

Dataset	Approximate Size
Small	1,000 documents
Medium	100,000 documents
Large	1,000,000+ documents

These are test configurations, not universal requirements.

The important point is to evaluate behavior beyond a tiny development dataset.

Keep the Database Configuration Consistent

The AI-generated query and expert-written query should run against the same:

Container
Partition key
Documents
Indexing configuration
Consistency configuration
Parameters
Database environment

The benchmark should look like:

             Same Requirement
                    |
          +---------+---------+
          |                   |
          v                   v
     Expert Query         AI Query
          |                   |
          +---------+---------+
                    |
                    v
             Same Cosmos DB
                    |
          +---------+---------+
          |                   |
          v                   v
       Results             RU Charge

Changing database configuration between tests makes the comparison unreliable.

Define the Ground Truth

Before generating an AI query, define what the correct result should contain.

For example:

Requirement:
Return the five most recent completed orders
for customer-42.

Expected:
- Only customer-42
- Only Completed orders
- Sorted newest first
- Maximum five results

This becomes the ground truth.

The AI-generated query can then be evaluated against it.

Test Query Correctness

Correctness should include more than syntax.

Check:

Filtering

Does the query return the correct customer?

Status

Does it return only the requested status?

Ordering

Are the results sorted correctly?

Result Count

Does it return no more than the requested number?

Projection

Does it return the required fields?

Null Handling

Does it behave correctly when optional properties are missing?

Boundary Conditions

Does date filtering behave correctly around the beginning and end of a range?

A query that executes successfully can still fail any of these tests.

Compare Results Programmatically

For repeatable testing, compare the results automatically.

A simple C# representation might be:

public sealed record OrderResult(
    string Id,
    string CustomerId,
    string Status,
    decimal Total,
    DateTime CreatedAt);

After executing both queries:

bool SameResults(
    IReadOnlyList<OrderResult> expected,
    IReadOnlyList<OrderResult> actual)
{
    return expected.SequenceEqual(actual);
}

In a real benchmark, the comparison should account for the required ordering and fields rather than relying on object equality alone.

Capture the RU Charge

Cosmos DB query responses expose the request charge.

For example:

using FeedIterator<OrderResult> iterator =
    container.GetItemQueryIterator<OrderResult>(
        queryDefinition);

while (iterator.HasMoreResults)
{
    FeedResponse<OrderResult> response =
        await iterator.ReadNextAsync();

    var requestCharge = response.RequestCharge;

    Console.WriteLine(
        $"RU Charge: {requestCharge}");
}

The benchmark should record the charge for every execution.

For a multi-page query, do not record only the first page.

Calculate the total:

Total RU =
Page 1 RU
+ Page 2 RU
+ Page 3 RU
+ ...

This gives a more realistic measurement of the complete query operation.

Measure RU Per Successful Task

A useful metric is:

RU per Successful Query

For example:

Query	Correct?	Total RU
Expert Query	Yes	Measure
AI Query	Yes	Measure

Do not compare RU consumption when one query returns incorrect data unless the result is explicitly categorized as a correctness failure.

Test Different Query Types

A benchmark should contain more than simple point-like filters.

Include categories such as:

Query Type	Example
Filtering	Find active orders
Sorting	Latest orders
Aggregation	Total sales
Projection	Return selected fields
Array filtering	Orders containing a product
Date filtering	Orders within a range
Pagination	Retrieve the next page
Multi-condition	Customer + status + date

Different query shapes can produce different behavior.

Test Partition-Key Awareness

Partitioning is one of the most important aspects of Cosmos DB query design.

Suppose the container uses:

/customerId

as its partition key.

A query that includes the partition key can be fundamentally different from one that does not.

For example:

SELECT *
FROM c
WHERE c.customerId = @customerId
AND c.status = @status

The AI should understand the data model well enough to use the relevant partition information when the application's requirements permit it.

The benchmark should therefore include:

Partition-Aware Query
vs.
Query Without Partition Context

and measure both correctness and RU behavior.

Do not assume that every query must contain the partition key. Some legitimate queries are intentionally broader.

The benchmark should evaluate whether the query matches the actual workload.

Test Queries With Different Selectivity

Data distribution matters.

Consider:

Customer A
10 orders

Customer B
10,000 orders

The same query structure may behave differently depending on the requested customer.

Test multiple parameter distributions:

No Matches
Few Matches
Typical Matches
Many Matches

This is more informative than testing a single customer.

Compare Projection Strategies

Suppose the requirement asks for:

Order ID
Status
Total

An AI system may generate:

SELECT *
FROM c
WHERE c.customerId = @customerId

while a more focused query might use:

SELECT
    c.id,
    c.status,
    c.total
FROM c
WHERE c.customerId = @customerId

The benchmark should compare the resulting behavior.

The point is not to assume that selecting fewer fields always produces a particular RU reduction. Measure the actual workload.

Test Pagination

Pagination is another useful benchmark category.

For example:

SELECT TOP @pageSize
    c.id,
    c.createdAt,
    c.total
FROM c
WHERE c.customerId = @customerId
ORDER BY c.createdAt DESC

The benchmark should test multiple pages and record:

Page Number
Rows Returned
RU Charge
Latency

A query that performs well for the first page may behave differently for later pages depending on the pagination strategy.

Test Aggregations

AI-generated aggregation queries deserve additional validation.

For example:

SELECT
    c.status,
    COUNT(1) AS orderCount
FROM c
WHERE c.customerId = @customerId
GROUP BY c.status

Verify:

Grouping
Filters
Result values
Null behavior
RU consumption

Aggregation queries can be particularly useful in a benchmark because correctness and resource consumption can both become important.

Test Array Queries

Cosmos DB documents often contain nested arrays.

For example:

{
  "id": "order-1001",
  "items": [
    {
      "productId": "p-100",
      "quantity": 2
    },
    {
      "productId": "p-200",
      "quantity": 1
    }
  ]
}

A requirement might be:

Find orders containing product p-100.

The generated query must correctly navigate the document structure.

A benchmark should verify:

Correct documents
+
Correct nested property
+
Correct filtering

This can reveal errors that are not visible in simple flat-document queries.

Evaluate Query Safety

AI-generated queries should pass through an application-level validation layer.

For example:

public sealed record QueryRequest(
    string Query,
    IReadOnlyDictionary<string, object> Parameters);

Before execution, validate:

Query
 |
 +-- Allowed container?
 +-- Allowed operation?
 +-- Parameterized?
 +-- Result limit?
 +-- Required filters?
 +-- Allowed properties?
 |
 v
Execute

The model should not be given unrestricted control over database execution.

Use Parameterized Queries

Avoid building queries through string concatenation.

Prefer:

var query =
    new QueryDefinition(
        "SELECT * FROM c " +
        "WHERE c.customerId = @customerId")
    .WithParameter(
        "@customerId",
        customerId);

This provides a cleaner boundary between generated query structure and user-provided values.

The AI can help construct the query, but parameter values should be handled by application code.

Create an Automated Benchmark Runner

A benchmark becomes much more useful when the complete process can be repeated.

A test case can be represented as:

public sealed record CosmosQueryTest(
    string Name,
    string Requirement,
    string ExpertQuery);

The AI query can then be generated independently and evaluated.

The benchmark result might contain:

public sealed record CosmosQueryResult(
    string Name,
    bool Correct,
    double TotalRu,
    double AverageLatencyMs,
    int ResultCount);

This allows the benchmark to compare many test cases consistently.

Repeat Each Query

Cosmos DB query behavior can vary with workload conditions.

Run each query multiple times where appropriate.

For example:

Test Case
 |
 +-- Expert Run 1
 +-- Expert Run 2
 +-- Expert Run 3
 |
 +-- AI Run 1
 +-- AI Run 2
 +-- AI Run 3

Record:

RU charge
Latency
Result count
Correctness

The benchmark should document whether each execution is performed under the same data and environment conditions.

Build a Query Quality Scorecard

A practical report might contain:

Category	Expert Query	AI Query
Correctness	Pass/Fail	Pass/Fail
Parameterization	Pass/Fail	Pass/Fail
Partition Awareness	Review	Review
Result Accuracy	Measure	Measure
RU Consumption	Measure	Measure
Latency	Measure	Measure
Maintainability	Review	Review

Do not reduce all of these dimensions to one number unless the scoring methodology is explicitly defined.

A query with slightly higher RU usage may still be acceptable if it is correct and maintainable.

Common Mistakes

Measuring RU Before Correctness

A cheap query that returns incorrect data is not a successful query.

Testing Only Small Datasets

Small datasets can hide scalability problems.

Using Different Database Configurations

The AI and expert queries must run against the same environment.

Ignoring Partitioning

Partition-key behavior can significantly affect query characteristics.

Comparing Only Average RU

A query may have unusual behavior for particular parameter values.

Giving the AI the Expert Query

This defeats the purpose of independent query generation.

Allowing Unvalidated Query Execution

AI-generated database operations should be controlled by application-level validation.

Treating RU as the Only Metric

Latency, correctness, result size, and maintainability also matter.

Troubleshooting

AI Query Returns Different Results

Compare the query components:

Filter
 |
 +-- Customer
 +-- Status
 +-- Date
 +-- Nested Properties
 |
Ordering
 |
Projection
 |
Pagination

Identify the first logical difference.

AI Query Uses More RUs

Check:

Partition-key usage
Result size
Filtering
Projection
Query structure
Data distribution
Number of returned pages

Then compare execution behavior under the same parameters.

AI Query Is Correct but Too Expensive

Do not immediately assume the query is unusable.

First determine why it consumes more RUs.

A query may require optimization because of:

Broad filtering
Missing partition context
Large result sets
Expensive aggregation
Poor pagination

AI Query Produces Invalid Syntax

Improve the schema and query-generation context supplied to the model.

Also validate generated queries before execution.

Results Change Between Runs

Check whether the underlying data is changing.

For deterministic benchmarking, use controlled data or capture the dataset state used by each run.

Best Practices

Define business requirements before generating queries.
Use a fixed benchmark dataset.
Keep database configuration identical between tests.
Validate correctness before comparing RU consumption.
Record total RU across all result pages.
Test different dataset sizes.
Test different parameter distributions.
Include partition-aware and broader queries.
Test nested documents and arrays.
Test aggregation and pagination.
Use parameterized queries.
Validate AI-generated queries before execution.
Record latency alongside RU consumption.
Repeat important benchmark cases.
Keep actual benchmark results separate from illustrative examples.

Advantages

Can reduce time spent writing exploratory queries.
Helps developers work with unfamiliar document structures.
Can accelerate database development workflows.
Provides a useful starting point for query optimization.
Can help identify query patterns that developers may want to refine manually.

Disadvantages

AI-generated queries can be logically incorrect.
Query quality depends on the schema and context provided.
Generated queries may consume more RUs than carefully optimized queries.
Complex document structures can be difficult for AI systems to interpret correctly.
AI-generated database operations still require validation and security controls.
Benchmark results can vary with data distribution and database configuration.

A Practical Evaluation Architecture

A production-oriented evaluation system can look like this:

                    Business Requirement
                            |
              +-------------+-------------+
              |                           |
              v                           v
       Expert Query                 AI Query
              |                           |
              +-------------+-------------+
                            |
                            v
                     Validation Layer
                            |
              +-------------+-------------+
              |             |             |
              v             v             v
         Correctness    Security      Query Rules
              |             |             |
              +-------------+-------------+
                            |
                            v
                     Azure Cosmos DB
                            |
              +-------------+-------------+
              |                           |
              v                           v
           Results                    RU Charge
              |                           |
              +-------------+-------------+
                            |
                            v
                     Benchmark Report

This architecture keeps the AI generation process separate from database enforcement.

Example Benchmark Matrix

A comprehensive benchmark can use:

Query Category	Small Data	Medium Data	Large Data	Edge Cases
Filtering	Test	Test	Test	Test
Sorting	Test	Test	Test	Test
Projection	Test	Test	Test	Test
Aggregation	Test	Test	Test	Test
Nested Arrays	Test	Test	Test	Test
Pagination	Test	Test	Test	Test
Partition-Aware	Test	Test	Test	Test

For each test, record:

Correctness
RU Consumption
Latency
Result Count
Query Validation

This creates a reusable evaluation framework rather than a one-time demonstration.

How to Interpret the Results

Suppose the benchmark produces:

Expert Query
Correct
RU: Measured

AI Query
Correct
RU: Measured

The next question is not simply:

Which number is lower?

Instead ask:

Is the RU difference meaningful?
Does it remain consistent across datasets?
Does it change with parameters?
Does the AI query remain correct?
Is the query maintainable?

Likewise, if the AI query is more efficient but produces incorrect results in certain edge cases, the efficiency advantage should not be treated as a success.

The benchmark should reveal trade-offs rather than force every result into a single ranking.

Conclusion

AI-assisted query generation can be valuable when working with Azure Cosmos DB, but generated queries should be evaluated like any other production database code. Successful execution is only the beginning.

A useful evaluation combines correctness, RU consumption, latency, partition behavior, result size, security, and maintainability. Testing these dimensions against the same database, schema, indexing configuration, and workload makes the comparison between AI-generated and expert-written queries much more meaningful.

The most practical approach is to use AI for what it does well: accelerating query creation and exploration, while keeping validation and execution controls in application code.

Ultimately, the question is not whether AI can generate a Cosmos DB query. It can. The more important question is whether the generated query is correct for the business requirement and efficient enough for the workload in which it will actually run.