Exploring Patterns in Azure Durable Functions

Azure Durable Functions is a serverless extension of Azure Functions that enables the creation of stateful and durable workflows. While it may not be immediately obvious, a storage account is a crucial component when running Azure Durable Functions. In this article, we will explore why a storage account is necessary for running Durable Functions and how it plays a fundamental role in the functioning of these workflows.

Durable Task Framework State Management
Azure Durable Functions relies on the Durable Task Framework (DTF), which is an open-source framework for building durable, stateful workflows. DTF requires a storage mechanism to persist the state of orchestrations and entities across executions. A storage account serves as this underlying storage layer. It ensures that the state of your orchestrations and entities is durable and can survive across restarts and failures.
Orchestration History
When you create an Azure Durable Function, the framework automatically stores the execution history and checkpoints of the orchestration. This history is crucial for the framework to maintain the state of the workflow, allowing it to resume from where it left off in case of interruptions or failures. The history data includes information about function calls, input and output values, and other relevant metadata.
Large Payloads and Output
Durable Functions often deal with input and output data that can be larger than the payload size limit of Azure Functions (around 100 MB). By using a storage account to store these payloads, Durable Functions can efficiently handle large data without encountering size constraints.
Distributed Locks and Coordination
In scenarios where Durable Functions need to coordinate and synchronize multiple instances or manage concurrency, a storage account is used for distributed locking and coordination. It ensures that only one instance of a function executes a particular task or acquires a lock at a time, preventing conflicts and race conditions.
Custom Stateful Entities
Durable Entities, a feature of Azure Durable Functions, allow you to create and manage stateful objects. These entities persist their state in a storage account, making them accessible and manageable across multiple function invocations. Without a storage account, the persistence and state management of these entities would be impossible.

Exploring Different Patterns in Azure Durable Functions

Azure Durable Functions is a powerful serverless extension to Azure Functions that allows you to build scalable, stateful, and event-driven workflows. With Durable Functions, you can create workflows that coordinate the execution of multiple functions in a reliable and efficient manner. In this article, we will explore some common patterns and use cases for Azure Durable Functions, along with code snippets to help you get started.

Prerequisites

Before we dive into the patterns, make sure you have the following prerequisites:

An Azure subscription.
Azure Functions tools and SDK installed (Azure Functions CLI).
A code editor (e.g., Visual Studio Code).
Basic knowledge of Azure Functions.

1. Chaining Functions

One of the fundamental patterns in Durable Functions is function chaining, where you execute a sequence of functions in a specific order. This pattern is useful for scenarios like data processing pipelines.

[FunctionName("OrchestrationFunction")]
public async Task<List<string>> RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var results = new List<string>();

    // Call Function1
    results.Add(await context.CallActivityAsync<string>("Function1", null));

    // Call Function2 with the result of Function1
    results.Add(await context.CallActivityAsync<string>("Function2", results[0]));

    // Call Function3 with the result of Function2
    results.Add(await context.CallActivityAsync<string>("Function3", results[1]));

    return results;
}

2. Fan-Out/Fan-In

This pattern is used when you need to execute multiple functions concurrently and then aggregate their results. It's useful for scenarios like parallel processing.

[FunctionName("FanOutFanInOrchestrator")]
public async Task<List<string>> RunFanOutFanInOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var tasks = new List<Task<string>>();

    for (int i = 0; i < 5; i++)
    {
        tasks.Add(context.CallActivityAsync<string>("ProcessData", i.ToString()));
    }

    await Task.WhenAll(tasks);

    var results = tasks.Select(t => t.Result).ToList();
    return results;
}

3. Human Interaction and Approval

You can use Durable Functions for human interaction, such as sending approval requests via email or Slack and waiting for a response before proceeding.

[FunctionName("ApprovalOrchestrator")]
public async Task<string> RunApprovalOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    // Send an approval request
    var approvalResponse = await context.WaitForExternalEvent<string>("ApprovalEvent");

    if (approvalResponse == "Approved")
    {
        // Proceed with the workflow
        return "Workflow Approved!";
    }
    else
    {
        // Handle rejection logic
        return "Workflow Rejected!";
    }
}

4. Monitoring and Timeout Handling

You can use Durable Functions to monitor the progress of long-running tasks and handle timeouts.

[FunctionName("TimeoutOrchestrator")]
public async Task<string> RunTimeoutOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var timeoutDuration = TimeSpan.FromMinutes(30);

    using (var cts = new CancellationTokenSource())
    {
        var timeoutTask = context.CreateTimer(context.CurrentUtcDateTime.Add(timeoutDuration), cts.Token);

        var resultTask = context.CallActivityAsync<string>("LongRunningTask", null);

        var winner = await Task.WhenAny(resultTask, timeoutTask);

        if (winner == resultTask)
        {
            // Task completed successfully
            cts.Cancel();
            return await resultTask;
        }
        else
        {
            // Handle timeout logic
            return "Task Timed Out!";
        }
    }
}

5. Stateful Entities

Durable Entities allow you to create and manage stateful objects that can be used across multiple function invocations.

[FunctionName("CounterEntity")]
public static void Run(
    [EntityTrigger] IDurableEntityContext ctx)
{
    var currentValue = ctx.GetState<int>();

    switch (ctx.OperationName)
    {
        case "add":
            var valueToAdd = ctx.GetInput<int>();
            currentValue += valueToAdd;
            break;
        case "get":
            ctx.Return(currentValue);
            break;
        case "reset":
            currentValue = 0;
            break;
    }

    ctx.SetState(currentValue);
}

Conclusion

Azure Durable Functions provide a wide range of patterns and capabilities for building reliable and scalable workflows. Whether you need to chain functions, handle human interactions, implement timeouts, or create stateful entities, Durable Functions can help you streamline your serverless applications. Experiment with these patterns and adapt them to your specific use cases to harness the full power of Azure Durable Functions in your projects.