How Microsoft Agent Framework Keeps an Agent Running After a Failure

Building an AI agent is relatively easy when every step succeeds. The real challenge starts when an agent performs a long-running task and something fails in the middle.

An agent might call an API, query a database, execute a tool, wait for an external system, and then continue with another step. If the application crashes after one of those operations, simply starting the agent again can create duplicate work or lose its progress.

This is where durable execution and workflow state become important.

The Microsoft Agent Framework provides concepts for building agent-based workflows that can preserve progress, recover from failures, and continue long-running operations instead of treating every execution as a completely new request.

This article explains the problem, how failure recovery works conceptually, and how you can design a C# agent workflow with reliability in mind.

Why Long-Running Agents Fail

A simple chatbot usually follows this pattern:

User
  |
  v
AI Model
  |
  v
Response

A long-running agent can be much more complicated:

User
  |
  v
Agent
  |
  +--> Search information
  |
  +--> Call external API
  |
  +--> Analyze result
  |
  +--> Update database
  |
  +--> Ask another agent
  |
  +--> Generate final response

Every external operation introduces another possible failure.

For example:

An API can return a timeout.
A database connection can fail.
A service can temporarily become unavailable.
The application process can restart.
A tool can return an unexpected response.
The machine running the workflow can become unavailable.

The biggest problem is not necessarily the failure itself. It is knowing where the workflow stopped.

Suppose an agent has completed three steps:

Step 1 - Read customer information       Completed
Step 2 - Validate account                 Completed
Step 3 - Create recommendation            Failed
Step 4 - Send notification                Not started

If the application restarts, the system should ideally continue from the appropriate point rather than executing Steps 1 and 2 again.

What Durable Execution Means

Durable execution means that the system can persist enough workflow state to resume execution after interruptions.

Conceptually, the workflow becomes:

Start
  |
  v
Step 1
  |
  v
Save progress
  |
  v
Step 2
  |
  v
Save progress
  |
  v
Step 3
  |
  X Failure
  |
  v
Resume
  |
  v
Step 3
  |
  v
Step 4

The important distinction is between application state and workflow progress.

Application state might contain customer records or configuration.

Workflow state answers questions such as:

Which step has completed?
Which operation is currently running?
What information must be carried into the next step?
What should happen after a failure?
Can a completed operation safely be skipped during recovery?

For long-running agents, these questions are essential.

Designing an Agent Workflow in C#

A useful approach is to separate the agent's workflow from the individual operations it performs.

For example:

public sealed class CustomerWorkflow
{
    public async Task RunAsync(
        string customerId,
        CancellationToken cancellationToken)
    {
        var customer = await LoadCustomerAsync(
            customerId,
            cancellationToken);

        var analysis = await AnalyzeCustomerAsync(
            customer,
            cancellationToken);

        await SaveRecommendationAsync(
            customerId,
            analysis,
            cancellationToken);

        await SendNotificationAsync(
            customerId,
            analysis,
            cancellationToken);
    }

    private Task<Customer> LoadCustomerAsync(
        string customerId,
        CancellationToken cancellationToken)
    {
        // Load customer information.
        throw new NotImplementedException();
    }

    private Task<Recommendation> AnalyzeCustomerAsync(
        Customer customer,
        CancellationToken cancellationToken)
    {
        // Call an agent or model.
        throw new NotImplementedException();
    }

    private Task SaveRecommendationAsync(
        string customerId,
        Recommendation recommendation,
        CancellationToken cancellationToken)
    {
        // Persist the recommendation.
        throw new NotImplementedException();
    }

    private Task SendNotificationAsync(
        string customerId,
        Recommendation recommendation,
        CancellationToken cancellationToken)
    {
        // Notify the user.
        throw new NotImplementedException();
    }
}

The example itself does not provide durability. It demonstrates an important architectural principle: make workflow steps explicit and independently recoverable.

A durable workflow system can then associate persisted state with these execution steps.

Why Checkpointing Matters

A checkpoint represents a known point of progress.

Consider an agent that processes a document:

1. Download document
2. Extract text
3. Summarize sections
4. Validate summary
5. Store result

If the process fails during step 4, restarting from step 1 may waste resources and repeat external operations.

A better workflow can preserve progress:

Download       Completed
Extract        Completed
Summarize      Completed
Validate       Failed
Store          Pending

After recovery:

Validate
   |
   v
Store

This becomes especially valuable when an agent performs expensive model calls or external operations.

Handling Transient Failures

Not every failure should stop an agent permanently.

Transient failures include situations such as:

temporary network problems
service throttling
short-lived database connectivity issues
temporary service unavailability

A retry policy can help.

A simple C# example looks like this:

public async Task<T> ExecuteWithRetryAsync<T>(
    Func<Task<T>> operation,
    int maxAttempts = 3)
{
    Exception? lastException = null;

    for (int attempt = 1; attempt <= maxAttempts; attempt++)
    {
        try
        {
            return await operation();
        }
        catch (Exception ex) when (attempt < maxAttempts)
        {
            lastException = ex;

            var delay = TimeSpan.FromSeconds(
                Math.Pow(2, attempt));

            await Task.Delay(delay);
        }
    }

    throw lastException!;
}

This uses exponential backoff:

Attempt 1 -> immediate
Attempt 2 -> short delay
Attempt 3 -> longer delay

In production, retries should be limited to failures that are actually retryable. Retrying every exception can make a failure worse.

Idempotency Is Critical

Recovery introduces another important concept: idempotency.

Suppose an agent sends a payment request:

await paymentService.CreatePaymentAsync(orderId);

If the operation succeeds but the application crashes before recording the result, the workflow might execute the payment operation again during recovery.

That can result in duplicate payments.

For operations with side effects, use an idempotency key where the external service supports it:

var request = new PaymentRequest
{
    OrderId = orderId,
    IdempotencyKey = workflowExecutionId
};

await paymentService.CreatePaymentAsync(request);

The exact implementation depends on the external system, but the principle remains the same:

A recovered workflow must be able to determine whether an external side effect has already happened.

Agent Failure vs Workflow Failure

These two failures should not be treated as the same thing.

Agent Failure

An AI model might return an invalid response or fail to complete a task.

The workflow can potentially:

Retry the agent call.
Ask the agent to correct its output.
Use another model or strategy.
Mark the step as failed.

Workflow Failure

The infrastructure running the workflow might stop unexpectedly.

For example:

Agent
  |
  +-- Tool call
  |
  +-- Application crashes
  |
  +-- Workflow state remains available
  |
  +-- Application restarts
  |
  +-- Workflow resumes

Durable execution primarily addresses the second problem while also helping structure recovery from the first.

Comparing Traditional Execution and Durable Execution

Area	Traditional Agent	Durable Workflow
Process restart	Usually starts again	Can resume persisted progress
Long-running tasks	Difficult to manage	Designed around workflow state
Failure recovery	Application-specific	Built around checkpoints/state
External side effects	Risk of duplication	Requires explicit idempotency design
Retries	Usually custom	Can be integrated into workflow steps
Debugging	Execution logs	Logs plus workflow state
Human interaction	More difficult	Easier to model as workflow steps

The exact capabilities depend on the framework components and persistence configuration you use, but the architectural difference is important.

Common Mistakes

Treating AI Agents Like Stateless API Calls

A short model request and a multi-hour workflow are very different problems.

If an agent can run for a long time, design for interruption from the beginning.

Retrying Every Exception

Retries are useful for transient failures but dangerous for permanent failures.

For example, retrying a validation error repeatedly does not fix the invalid input.

Ignoring External Side Effects

Database writes, emails, payments, messages, and API operations can produce effects outside the workflow.

Always consider what happens if the process crashes immediately after the external operation succeeds.

Storing Everything in Memory

An in-memory variable disappears when the process terminates.

Important workflow state should have an appropriate persistence strategy.

Making One Huge Agent Step

A workflow such as this:

RunEverything()

is difficult to recover.

Prefer meaningful steps:

LoadData()
AnalyzeData()
ValidateResult()
SaveResult()
NotifyUser()

Smaller boundaries make recovery and troubleshooting easier.

Best Practices for Production

Keep Workflow Steps Small

Each step should represent a meaningful unit of work.

This makes failures easier to identify and recovery easier to reason about.

Make Side Effects Idempotent

Use request identifiers, unique database constraints, or idempotency keys when appropriate.

Persist Important State

Do not rely on process memory for information required after a restart.

Separate Retryable and Permanent Errors

Classify failures before deciding whether to retry.

Log Execution Context

Useful logs should include information such as:

Workflow ID
Step name
Attempt number
Start time
Completion time
Failure reason

Avoid logging sensitive customer or authentication information.

Design for Cancellation

Long-running agents should respect cancellation:

public async Task RunAsync(
    CancellationToken cancellationToken)
{
    cancellationToken.ThrowIfCancellationRequested();

    await ExecuteStepAsync(cancellationToken);
}

Cancellation is different from failure. A user or system may intentionally stop a workflow, and the application should handle that state explicitly.

Troubleshooting Agent Recovery

When a workflow does not resume correctly, check these areas:

Was workflow state actually persisted?
Can the workflow identify its previous execution?
Was the failed step retryable?
Did an external side effect already occur?
Is the operation idempotent?
Are retries hiding the original error?
Does the workflow distinguish cancellation from failure?
Do logs contain the workflow and step identifiers?

These checks usually reveal whether the problem is related to persistence, retry behavior, or an external dependency.

Advantages and Disadvantages

Advantages

Better recovery from application interruptions
More reliable long-running agent workflows
Reduced need to restart completed work
Clearer workflow state
Easier handling of retries and failures
Better foundation for human-in-the-loop workflows

Disadvantages

More architectural complexity than a simple agent call
Requires careful state management
External side effects still require idempotency
Persistence adds operational considerations
Debugging requires understanding both agent behavior and workflow execution

Final Thoughts

A production AI agent should not be designed with the assumption that every operation will finish successfully.

Long-running agents need a workflow model that understands progress, failure, retries, persistence, and recovery. Microsoft Agent Framework can be used as part of this architecture by structuring agent operations as workflow steps rather than treating the entire task as one disposable execution.

The most important lesson is that durable execution is not just about restarting an agent. It is about knowing what has already happened, what still needs to happen, and how to safely continue without repeating harmful side effects.

When building a C# agent that may run for minutes, hours, or longer, design recovery and idempotency alongside the agent itself rather than adding them after the first production failure.

Building an AI agent is relatively easy when every step succeeds. The real challenge starts when an agent performs a long-running task and something fails in the middle.

This is where durable execution and workflow state become important.

This article explains the problem, how failure recovery works conceptually, and how you can design a C# agent workflow with reliability in mind.

Why Long-Running Agents Fail

A simple chatbot usually follows this pattern:

User
  |
  v
AI Model
  |
  v
Response

A long-running agent can be much more complicated:

User
  |
  v
Agent
  |
  +--> Search information
  |
  +--> Call external API
  |
  +--> Analyze result
  |
  +--> Update database
  |
  +--> Ask another agent
  |
  +--> Generate final response

Every external operation introduces another possible failure.

For example:

An API can return a timeout.
A database connection can fail.
A service can temporarily become unavailable.
The application process can restart.
A tool can return an unexpected response.
The machine running the workflow can become unavailable.

The biggest problem is not necessarily the failure itself. It is knowing where the workflow stopped.

Suppose an agent has completed three steps:

Step 1 - Read customer information       Completed
Step 2 - Validate account                 Completed
Step 3 - Create recommendation            Failed
Step 4 - Send notification                Not started

If the application restarts, the system should ideally continue from the appropriate point rather than executing Steps 1 and 2 again.

What Durable Execution Means

Durable execution means that the system can persist enough workflow state to resume execution after interruptions.

Conceptually, the workflow becomes:

Start
  |
  v
Step 1
  |
  v
Save progress
  |
  v
Step 2
  |
  v
Save progress
  |
  v
Step 3
  |
  X Failure
  |
  v
Resume
  |
  v
Step 3
  |
  v
Step 4

The important distinction is between application state and workflow progress.

Application state might contain customer records or configuration.

Workflow state answers questions such as:

Which step has completed?
Which operation is currently running?
What information must be carried into the next step?
What should happen after a failure?
Can a completed operation safely be skipped during recovery?

For long-running agents, these questions are essential.

Designing an Agent Workflow in C#

A useful approach is to separate the agent's workflow from the individual operations it performs.

For example:

public sealed class CustomerWorkflow
{
    public async Task RunAsync(
        string customerId,
        CancellationToken cancellationToken)
    {
        var customer = await LoadCustomerAsync(
            customerId,
            cancellationToken);

        var analysis = await AnalyzeCustomerAsync(
            customer,
            cancellationToken);

        await SaveRecommendationAsync(
            customerId,
            analysis,
            cancellationToken);

        await SendNotificationAsync(
            customerId,
            analysis,
            cancellationToken);
    }

    private Task<Customer> LoadCustomerAsync(
        string customerId,
        CancellationToken cancellationToken)
    {
        // Load customer information.
        throw new NotImplementedException();
    }

    private Task<Recommendation> AnalyzeCustomerAsync(
        Customer customer,
        CancellationToken cancellationToken)
    {
        // Call an agent or model.
        throw new NotImplementedException();
    }

    private Task SaveRecommendationAsync(
        string customerId,
        Recommendation recommendation,
        CancellationToken cancellationToken)
    {
        // Persist the recommendation.
        throw new NotImplementedException();
    }

    private Task SendNotificationAsync(
        string customerId,
        Recommendation recommendation,
        CancellationToken cancellationToken)
    {
        // Notify the user.
        throw new NotImplementedException();
    }
}

The example itself does not provide durability. It demonstrates an important architectural principle: make workflow steps explicit and independently recoverable.

A durable workflow system can then associate persisted state with these execution steps.

Why Checkpointing Matters

A checkpoint represents a known point of progress.

Consider an agent that processes a document:

1. Download document
2. Extract text
3. Summarize sections
4. Validate summary
5. Store result

If the process fails during step 4, restarting from step 1 may waste resources and repeat external operations.

A better workflow can preserve progress:

Download       Completed
Extract        Completed
Summarize      Completed
Validate       Failed
Store          Pending

After recovery:

Validate
   |
   v
Store

This becomes especially valuable when an agent performs expensive model calls or external operations.

Handling Transient Failures

Not every failure should stop an agent permanently.

Transient failures include situations such as:

temporary network problems
service throttling
short-lived database connectivity issues
temporary service unavailability

A retry policy can help.

A simple C# example looks like this:

public async Task<T> ExecuteWithRetryAsync<T>(
    Func<Task<T>> operation,
    int maxAttempts = 3)
{
    Exception? lastException = null;

    for (int attempt = 1; attempt <= maxAttempts; attempt++)
    {
        try
        {
            return await operation();
        }
        catch (Exception ex) when (attempt < maxAttempts)
        {
            lastException = ex;

            var delay = TimeSpan.FromSeconds(
                Math.Pow(2, attempt));

            await Task.Delay(delay);
        }
    }

    throw lastException!;
}

This uses exponential backoff:

Attempt 1 -> immediate
Attempt 2 -> short delay
Attempt 3 -> longer delay

In production, retries should be limited to failures that are actually retryable. Retrying every exception can make a failure worse.

Idempotency Is Critical

Recovery introduces another important concept: idempotency.

Suppose an agent sends a payment request:

await paymentService.CreatePaymentAsync(orderId);

If the operation succeeds but the application crashes before recording the result, the workflow might execute the payment operation again during recovery.

That can result in duplicate payments.

For operations with side effects, use an idempotency key where the external service supports it:

var request = new PaymentRequest
{
    OrderId = orderId,
    IdempotencyKey = workflowExecutionId
};

await paymentService.CreatePaymentAsync(request);

The exact implementation depends on the external system, but the principle remains the same:

A recovered workflow must be able to determine whether an external side effect has already happened.

Agent Failure vs Workflow Failure

These two failures should not be treated as the same thing.

Agent Failure

An AI model might return an invalid response or fail to complete a task.

The workflow can potentially:

Retry the agent call.
Ask the agent to correct its output.
Use another model or strategy.
Mark the step as failed.

Workflow Failure

The infrastructure running the workflow might stop unexpectedly.

For example:

Agent
  |
  +-- Tool call
  |
  +-- Application crashes
  |
  +-- Workflow state remains available
  |
  +-- Application restarts
  |
  +-- Workflow resumes

Durable execution primarily addresses the second problem while also helping structure recovery from the first.

Comparing Traditional Execution and Durable Execution

Area	Traditional Agent	Durable Workflow
Process restart	Usually starts again	Can resume persisted progress
Long-running tasks	Difficult to manage	Designed around workflow state
Failure recovery	Application-specific	Built around checkpoints/state
External side effects	Risk of duplication	Requires explicit idempotency design
Retries	Usually custom	Can be integrated into workflow steps
Debugging	Execution logs	Logs plus workflow state
Human interaction	More difficult	Easier to model as workflow steps

The exact capabilities depend on the framework components and persistence configuration you use, but the architectural difference is important.

Common Mistakes

Treating AI Agents Like Stateless API Calls

A short model request and a multi-hour workflow are very different problems.

If an agent can run for a long time, design for interruption from the beginning.

Retrying Every Exception

Retries are useful for transient failures but dangerous for permanent failures.

For example, retrying a validation error repeatedly does not fix the invalid input.

Ignoring External Side Effects

Database writes, emails, payments, messages, and API operations can produce effects outside the workflow.

Always consider what happens if the process crashes immediately after the external operation succeeds.

Storing Everything in Memory

An in-memory variable disappears when the process terminates.

Important workflow state should have an appropriate persistence strategy.

Making One Huge Agent Step

A workflow such as this:

RunEverything()

is difficult to recover.

Prefer meaningful steps:

LoadData()
AnalyzeData()
ValidateResult()
SaveResult()
NotifyUser()

Smaller boundaries make recovery and troubleshooting easier.

Best Practices for Production

Keep Workflow Steps Small

Each step should represent a meaningful unit of work.

This makes failures easier to identify and recovery easier to reason about.

Make Side Effects Idempotent

Use request identifiers, unique database constraints, or idempotency keys when appropriate.

Persist Important State

Do not rely on process memory for information required after a restart.

Separate Retryable and Permanent Errors

Classify failures before deciding whether to retry.

Log Execution Context

Useful logs should include information such as:

Workflow ID
Step name
Attempt number
Start time
Completion time
Failure reason

Avoid logging sensitive customer or authentication information.

Design for Cancellation

Long-running agents should respect cancellation:

public async Task RunAsync(
    CancellationToken cancellationToken)
{
    cancellationToken.ThrowIfCancellationRequested();

    await ExecuteStepAsync(cancellationToken);
}

Cancellation is different from failure. A user or system may intentionally stop a workflow, and the application should handle that state explicitly.

Troubleshooting Agent Recovery

When a workflow does not resume correctly, check these areas:

Was workflow state actually persisted?
Can the workflow identify its previous execution?
Was the failed step retryable?
Did an external side effect already occur?
Is the operation idempotent?
Are retries hiding the original error?
Does the workflow distinguish cancellation from failure?
Do logs contain the workflow and step identifiers?

These checks usually reveal whether the problem is related to persistence, retry behavior, or an external dependency.

Advantages and Disadvantages

Advantages

Better recovery from application interruptions
More reliable long-running agent workflows
Reduced need to restart completed work
Clearer workflow state
Easier handling of retries and failures
Better foundation for human-in-the-loop workflows

Disadvantages

More architectural complexity than a simple agent call
Requires careful state management
External side effects still require idempotency
Persistence adds operational considerations
Debugging requires understanding both agent behavior and workflow execution

Final Thoughts

A production AI agent should not be designed with the assumption that every operation will finish successfully.

When building a C# agent that may run for minutes, hours, or longer, design recovery and idempotency alongside the agent itself rather than adding them after the first production failure.