Document processing is one of those workloads that looks simple until it reaches production.
A basic proof of concept may only need to extract a few fields from a PDF. A production system, however, may need to process invoices, contracts, forms, emails, spreadsheets, scanned documents, and other files while preserving structure, validating extracted values, handling failures, and sending uncertain results for human review.
Azure Content Understanding is designed for this broader problem. It combines document and multimodal content extraction with generative AI-based understanding and can produce structured representations suitable for downstream applications, retrieval, and agent workflows. Microsoft describes it as a content AI service that supports documents, images, audio, and video.
Recent platform updates also add capabilities such as synchronous Read and Layout processing, agentic understanding, broader file support, and expanded GPT-5-family model support. Because these capabilities and API versions can evolve, production implementations should pin API versions and validate behavior against an evaluation dataset before changing live workloads.
This article focuses on how to design a production-oriented document pipeline rather than treating Content Understanding as simply another OCR endpoint.
What Is Azure Content Understanding?
Traditional document-processing systems generally follow a pipeline like this:
Document
|
v
OCR
|
v
Text Extraction
|
v
Rules / Regex
|
v
Structured Data
This works well for predictable documents, but becomes difficult when layouts, terminology, languages, tables, handwriting, or document structures vary.
A modern Content Understanding pipeline can instead look like this:
+------------------+
| Incoming Document|
+--------+---------+
|
v
+---------------------+
| Content Understanding|
+----------+----------+
|
+--------------+--------------+
| | |
v v v
Layout Fields Figures
| | |
+--------------+--------------+
|
v
Structured Output
|
+--------------+--------------+
| | |
v v v
RAG Workflow Agent
The important architectural difference is that the system attempts to preserve the meaning and structure of the source rather than reducing everything to plain OCR text.
Microsoft's GA announcement describes Content Understanding as combining specialized models for capabilities such as OCR and layout with generative AI for tasks such as field extraction, segmentation, and figure analysis.
Why Production Pipelines Need More Than Extraction
A production document pipeline usually has several responsibilities beyond extracting fields.
For example, an invoice-processing system may need to:
Accept an uploaded document.
Validate the file.
Store the original document.
Analyze the content.
Extract structured fields.
Validate important values.
Assign a confidence or review state.
Persist the extracted result.
Send low-confidence documents for human review.
Publish validated data to downstream systems.
A useful architecture is therefore:
Client
|
v
Upload API
|
+----> Blob/Object Storage
|
v
Processing Queue
|
v
Content Understanding
|
v
Validation
|
+----> Accepted
|
+----> Human Review
|
+----> Failed / Retry
|
v
Application Database
|
v
Business Workflow
This separation is important because document analysis should not become tightly coupled to the HTTP request that originally uploaded the file.
Supported Document Workloads
Content Understanding has expanded beyond traditional PDF-centric processing. Microsoft has documented support for additional formats including .eml, .msg, legacy Office formats, and OpenDocument formats, as well as extraction of embedded figures from Office documents.
This matters when designing ingestion systems.
Instead of forcing every file through a conversion service first, the pipeline can use native processing where supported.
For example:
| Input | Potential Processing |
|---|
| PDF | Read, layout, extraction |
| Scanned document | OCR + understanding |
| DOCX | Structure + text + figures |
| XLSX | Content and embedded figures |
| EML | Message/document processing |
| MSG | Message/document processing |
| Image | Visual understanding |
| Audio | Transcription and understanding |
| Video | Multimodal processing |
The exact analyzer and supported capabilities should always be checked against the API version and region used by the application.
Designing the Ingestion Layer
The ingestion layer should be deliberately independent from the AI processing layer.
A simplified ASP.NET Core endpoint could look like this:
[ApiController]
[Route("api/documents")]
public class DocumentsController : ControllerBase
{
[HttpPost]
public async Task<IActionResult> Upload(
IFormFile file,
CancellationToken cancellationToken)
{
if (file is null || file.Length == 0)
{
return BadRequest("A document is required.");
}
var documentId = Guid.NewGuid();
// Store the original document.
// Persist metadata required for asynchronous processing.
return Accepted(new
{
DocumentId = documentId,
Status = "Queued"
});
}
}
Returning 202 Accepted is often more appropriate than keeping the HTTP request open while a potentially expensive document operation completes.
The API should record metadata such as:
public sealed class DocumentRecord
{
public Guid Id { get; set; }
public string FileName { get; set; } = "";
public string ContentType { get; set; } = "";
public long SizeBytes { get; set; }
public string Status { get; set; } = "Queued";
public DateTimeOffset CreatedAt { get; set; }
}
The original file should also be retained according to the application's retention and compliance requirements.
Calling Content Understanding
Microsoft provides REST-based quickstarts for submitting files to prebuilt analyzers. The documented request pattern uses an analyzer endpoint and supports document, image, audio, and video workloads.
A simplified HTTP request might look like:
POST {endpoint}/contentunderstanding/analyzers/prebuilt-invoice:analyze
Content-Type: application/octet-stream
Ocp-Apim-Subscription-Key: {key}
<document bytes>
For production applications, avoid embedding API keys directly in source code.
A managed identity or another supported identity mechanism is generally preferable where the service and deployment architecture support it.
The important design principle is to isolate the Content Understanding call behind an application service:
public interface IDocumentAnalyzer
{
Task<DocumentAnalysisResult> AnalyzeAsync(
Stream document,
string contentType,
CancellationToken cancellationToken);
}
Your business logic can then depend on IDocumentAnalyzer rather than directly depending on a particular HTTP endpoint.
Using Prebuilt Analyzers
Prebuilt analyzers are useful when the document type matches an existing scenario.
For example:
Invoice
|
v
prebuilt-invoice
|
v
Vendor
Invoice Number
Invoice Date
Total
Line Items
This reduces the amount of custom extraction logic your application needs to maintain.
Microsoft also documents domain-specific analyzers for scenarios including finance, contracts and procurement, mortgage and lending, and identity verification.
Before building a custom analyzer, determine whether a prebuilt analyzer already provides the required structure.
Designing Custom Schemas
Some business documents do not map cleanly to a prebuilt analyzer.
For example, a purchase contract might require:
{
"contractNumber": "",
"supplier": "",
"effectiveDate": "",
"expirationDate": "",
"renewalTerms": "",
"paymentTerms": "",
"terminationClauses": []
}
The production concern is not simply extracting these values. You also need to define what happens when a field is missing, ambiguous, or inconsistent with the source document.
For critical fields, treat extracted values as data requiring validation, not automatically trusted business facts.
Semantic and Layout-Aware Processing
Document structure matters.
Consider:
Total Amount
$12,500
versus:
Subtotal $10,000
Tax $2,500
Total $12,500
Plain text extraction can lose relationships between labels, values, tables, headings, and surrounding content.
Microsoft has highlighted layout-aware Markdown and richer structural representations as ways to improve downstream grounding and retrieval.
This becomes particularly important when Content Understanding output feeds a RAG system.
A pipeline can preserve:
Document
|
+-- Heading
+-- Paragraph
+-- Table
+-- Figure
+-- Caption
+-- Page / location metadata
instead of producing one large text string.
Building a Validation Layer
Never make the AI extraction layer the final authority for important business decisions.
Suppose the extracted invoice total is:
{
"subtotal": 10000,
"tax": 2500,
"total": 13000
}
A deterministic validation rule can identify an inconsistency:
bool IsValidInvoiceTotal(
decimal subtotal,
decimal tax,
decimal total)
{
return Math.Abs((subtotal + tax) - total) < 0.01m;
}
This is an important production pattern:
Use AI for interpretation and deterministic code for rules that can be expressed deterministically.
The same approach can be applied to dates, currency values, identifiers, totals, and required fields.
Confidence and Human Review
Not every document should follow the same path.
A useful processing state machine is:
+---------+
| Queued |
+----+----+
|
v
+---------+
| Analyzing|
+----+----+
|
v
+-----------+
| Validating|
+-----+-----+
|
+--------+--------+
| |
v v
Accepted Review
| |
v v
Complete Human Action
Microsoft documents confidence scores and grounding support as part of Content Understanding's extraction capabilities.
Confidence should not be treated as a universal guarantee of correctness. Instead, define application-specific review policies.
For example:
High confidence + validation passed
-> Automatic processing
Low confidence
-> Human review
Validation failed
-> Human review
Processing error
-> Retry / failure queue
Asynchronous vs Synchronous Processing
Traditional document processing often uses an asynchronous operation:
Submit
|
v
Operation ID
|
v
Poll
|
v
Result
Microsoft announced synchronous Read and Layout APIs as part of the newer Content Understanding direction, allowing results to be obtained without managing the traditional asynchronous workflow for supported scenarios.
That does not mean every production pipeline should become synchronous.
Use synchronous processing when:
Documents are small.
The request must return quickly.
The supported operation fits the API behavior.
Failure handling is straightforward.
Use asynchronous processing when:
Documents can be large.
Processing may take longer.
Throughput matters.
Retries need to be independent of client requests.
Human review is part of the workflow.
For many enterprise systems, asynchronous ingestion remains the safer architectural boundary.
Agentic Document Processing
Content Understanding is also being integrated with agent-oriented workflows.
Microsoft has documented integration with Microsoft Agent Framework, Foundry IQ, LangChain, and other tools. In the Agent Framework scenario, an agent can use Content Understanding when it needs to interpret a document or image.
This enables an architecture such as:
User
|
v
Agent
|
+----> Document Understanding
|
+----> Knowledge Retrieval
|
+----> Business Tools
|
v
Action
However, adding an agent does not eliminate the need for validation.
For financial, legal, compliance, or operational workflows, the agent should operate inside explicit boundaries and should not be allowed to convert uncertain extraction directly into irreversible actions.
Retry and Failure Handling
Document pipelines must expect transient failures.
A worker should distinguish between:
Transient Failure
-> Retry
Permanent Input Error
-> Reject
Unsupported Format
-> Reject / Conversion Workflow
Validation Failure
-> Human Review
Authentication / Configuration Error
-> Alert
Avoid infinite retries.
A simple policy can use exponential backoff:
var delay = TimeSpan.FromSeconds(
Math.Pow(2, attempt));
await Task.Delay(delay, cancellationToken);
In a real production system, use a resilience library or centralized retry policy rather than implementing retry behavior independently in every worker.
Observability
A document pipeline should make every processing stage observable.
Track fields such as:
| Metric | Why It Matters |
|---|
| Documents received | Workload volume |
| Processing duration | Performance |
| Failure rate | Reliability |
| Retry count | Dependency health |
| Analyzer used | Routing visibility |
| Model used | Cost and quality analysis |
| Review rate | Automation effectiveness |
| Validation failure rate | Extraction reliability |
| Processing cost | Financial control |
Use a correlation ID throughout the pipeline:
Request
|
Correlation ID
|
Upload
|
Queue
|
Analyzer
|
Validation
|
Database
This makes troubleshooting considerably easier than trying to correlate logs using filenames or timestamps.
Production Security Considerations
Documents can contain highly sensitive business information.
Important controls include:
Encrypt documents at rest and in transit.
Use managed identity where appropriate.
Avoid logging raw document contents.
Avoid logging extracted sensitive fields unnecessarily.
Apply least-privilege access.
Define document retention policies.
Control access to human-review interfaces.
Validate file type and size before processing.
Track who accessed or modified extracted results.
Evaluate regional processing requirements before deployment.
Microsoft has also documented global and data-zone processing options as part of Content Understanding's evolving platform capabilities.
Availability and supported deployment options should be verified for the specific Azure region and API version selected by your application.
Advantages
Supports structured understanding beyond basic OCR.
Can process multiple content modalities.
Provides prebuilt analyzers for common document scenarios.
Supports custom extraction scenarios.
Can produce richer, layout-aware representations.
Integrates with retrieval and agent workflows.
Can reduce the amount of custom document-processing code.
Supports model selection through Microsoft Foundry deployments.
Disadvantages
AI-based extraction can still produce incorrect results.
Processing behavior can change when models or analyzer configurations change.
Complex workloads require evaluation and validation.
Consumption-based processing introduces operational cost.
Production systems still require queues, retries, observability, and security controls.
Some capabilities depend on API version, region, and service availability.
Common Mistakes
Treating Extraction as Ground Truth
Extracted data should be validated before it drives important business actions.
Processing Everything Synchronously
Long-running document operations should not unnecessarily block an HTTP request.
Logging Complete Documents
Application logs are not an appropriate place to store raw confidential documents.
Skipping Evaluation
A new model or analyzer configuration can change accuracy and latency. Microsoft specifically recommends side-by-side evaluation before moving new model configurations into production.
Building Custom OCR Before Checking Available Analyzers
Evaluate the prebuilt and platform capabilities first. Custom processing should solve a demonstrated gap rather than duplicate an existing service capability.
Troubleshooting
Processing Is Taking Too Long
Check:
Separate analyzer latency from queue latency so the actual bottleneck is visible.
Extraction Quality Is Poor
Start by examining the source documents.
Check whether the problem is caused by:
Use a representative evaluation dataset rather than judging the system from one document.
Valid Fields Are Being Rejected
Inspect the deterministic validation layer separately from AI extraction.
A correct extraction can still fail if business validation rules are too strict.
Duplicate Documents Are Being Processed
Introduce an idempotency strategy.
For example, generate a content hash:
using System.Security.Cryptography;
using var sha256 = SHA256.Create();
var hash = await sha256.ComputeHashAsync(
documentStream,
cancellationToken);
var documentHash = Convert.ToHexString(hash);
Store the hash with the document metadata and use it as part of the duplicate-detection strategy where appropriate.
Best Practices
Separate ingestion, analysis, validation, and business processing.
Store the original document independently from extracted data.
Use asynchronous processing for workloads that do not require immediate results.
Prefer prebuilt analyzers when they satisfy the business requirement.
Preserve document structure when downstream RAG or agent workflows need it.
Validate important extracted values with deterministic rules.
Create an explicit human-review path.
Implement bounded retries and dead-letter handling.
Track analyzer, model, API version, and configuration for every processing run.
Avoid logging sensitive document contents.
Maintain a representative evaluation dataset.
Run side-by-side evaluations before changing production models or analyzers.
Monitor processing cost and latency continuously.
Design the pipeline so the AI service can be replaced or upgraded without rewriting business logic.
Frequently Asked Questions
Is Azure Content Understanding the same as OCR?
No. OCR is primarily concerned with recognizing text. Content Understanding combines document and multimodal processing with richer structural and generative-AI-based understanding capabilities. Microsoft describes it as combining specialized document processing with generative AI for extraction and content reasoning.
Can Content Understanding process invoices?
Yes. Microsoft provides prebuilt analyzers for document scenarios including invoices, and its quickstart demonstrates submitting a document to the prebuilt-invoice analyzer.
Should I use a custom analyzer for every document type?
No. Start with the available prebuilt analyzer that matches the workload. Move to a custom analyzer when the application's schema or extraction requirements are not adequately covered.
Can the output be used for RAG?
Yes. Microsoft has specifically positioned Content Understanding as a way to generate richer representations for retrieval workflows, including layout-aware Markdown and integration with Azure AI Search and Foundry IQ.
Should document processing always use an AI model?
Not necessarily. Deterministic processing remains valuable for straightforward transformations and validation. A production architecture should use AI where semantic interpretation is required and traditional code where deterministic logic is sufficient.
How should low-confidence documents be handled?
Do not automatically accept them. Route them to a human-review workflow or another controlled verification process based on the business risk associated with the extracted information.
Conclusion
A production document pipeline is much more than an API call that extracts text from a PDF. The real engineering challenge is building a reliable workflow around ingestion, content analysis, validation, retries, observability, security, and human review.
Azure Content Understanding provides a broader foundation for this architecture by combining document and multimodal processing with structured extraction, layout-aware content, generative AI, and integrations with retrieval and agent workflows.
The strongest production design is therefore not simply "send every document to an AI model." It is a controlled pipeline in which Content Understanding performs the interpretation work, deterministic application code validates critical information, and the surrounding platform handles reliability, security, monitoring, and business workflows.
That separation makes the system easier to operate today and easier to evolve as document formats, models, analyzers, and AI capabilities continue to change.