Introducing Durable Patterns for Azure Durable Functions

Azure Functions: a brief introduction

Azure Functions is a serverless approach provided by Microsoft. It is similar to AWS Lambda and Google Cloud Functions, providing a way to run code without worrying about resource management or provisioning. Users can run event-triggered code and pay only for what was used.

Durable Function Framework: a quick overview

Durable functions are extensions to Azure Functions that introduce state to serverless computing. They allow you to write stateful functions and workflows by introducing orchestrator and entity functions. There are several most common use cases of the framework that can be described by patterns:

• Function chaining - where we run one activity function after another

• Fan-out/fan-in - running multiple activity functions in parallel and waiting for the results

• Human interaction

• Async HTTP APIs

• Monitoring

• Aggregator (stateful entities)

How to make Durable Functions better?

Durable Function Framework is an extremely useful tool for running serverless workflows, but it can be improved. The need to create function classes, use many attributes, and generate a lot of boilerplate code means that the Durable Function SDK has significant room for improvement.

Durable Patterns to the rescue

Durable Patterns is a .NET library that simplifies usage of the Durable Function framework when leveraging main use patterns. It provides fluent interfaces to wrap business logic and, under the hood, it uses activity functions, async code and batching to make the best use of the durable framework. Developers can focus on writing business code without worrying about boilerplate. The fluent interface makes it easy to chain different pattern usages one after another in an intuitive way.

Let’s look at an example (the diagram at the top).

Function chaining with fan-out/fan-in

The following example shows an http trigger which starts the flow. It uses the GetItems function to get a list of items, then the ProcessItem function that works on every item. Each item is processed by one function for simplicity (Durable Patterns supports batching). After the parallel processing, the ProcessResults function works on the result list.

Normally, we would need to create an orchestrator function like the one shown below:

[FunctionName("Orchestrator")]
public static async Task Run([OrchestrationTrigger] IDurableOrchestrationContext context)
{ 
  var parallelTasks = new List<Task<object>>(); object[] items = await context.CallActivityAsync<object[]>("GetItems", null); for (int i = 0; i < items.Length; i++) 
  { 
    Task<object> task = context.CallActivityAsync<object>("ProcessItem", items[i]); 
    parallelTasks.Add(task);
  } 
  var results = await Task.WhenAll(parallelTasks); 
  await context.CallActivityAsync<int>("ProcessResults", results

Then, we need to write all the activity functions, either in separate classes (boilerplate) or in the same (messy), remembering all the attributes, etc.

[FunctionName("GetItems")] 
public Task<object[]> GetItems([ActivityTrigger] IDurableActivityContext context)
{ 
  return _service1.GetItemsAsync(); 
}

[FunctionName("ProcessItem")] 
public Task<object> ProcessItem([ActivityTrigger] IDurableActivityContext context, object item)
{ 
  return _service2.ProcessItemAsync(item); 
}

[FunctionName("ProcessResults")] 
public Task ProcessResults([ActivityTrigger] IDurableActivityContext context, object[] results)
{ 
return _service3.ProcessResultsAsync(results

We would also need to inject all the dependencies needed by any of the class to the Orchestrator class itself. It doesn’t look very dev friendly, right?

Luckily, we have a solution: Durable Patterns.

With Durable Patterns, we can easily define our Azure Functions using a fluent syntax that allows us to inject our dependencies without the need to modify the function signature.

Using Durable Patterns

Here is an example of how we can use Durable Patterns to simplify our code:

[FunctionName("Orchestrator")] 
public async Task RunOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context) 
{ 
  await _patterns 
    .WithContext(context) 
    .RunActivity<GetItemsActivity>() 
    .FanOutFanIn<ProcessItemActivity>(new FanOutFanInOptions( 
      BatchSize: 1, 
      ParallelActivityFunctionsCap: 5)) 
    .RunActivity<ProcessResultsActivity>() 
    .ExecuteAsync

As we can see, we can now define our function in a single line using the RunActivity and FanOutFanIn methods. This syntax is much more concise and easier to read than the previous example.

Pattern activity classes

In the example above, we are using activity classes such as GetItemsActivity or ProcessItemActivity - these are defined by the user and encapsulate process logic. This is how GetItemsActivity class could look like:

internal class GetItemsActivity : IPatternActivity<List<object>>
{ 
  private readonly IItemRepository _repository; 
  private readonly ILogger<GetItemsActivity> _logger; 
  
  public GetItemsActivity( 
      IItemRepository repository, 
      ILogger<GetItemsActivity> logger)
{ 
  _repository = repository;
  _logger = logger; 
} 

  public Task<List<object>> RunAsync(object? input)
  { 
  _logger.LogInformation("getting items from repository"); return _repository.GetItemsAsync

The logic in ProcessItemActivity class will be executed in a fan-out-fan-in manner, which means the collection returned from the GetItemsActivity step will be chunked into batches (in this example of size 1) and the ProcessItemActivity.RunAsync will be executed in parallel (in a separate activity function) for each of these batches. An implementation of this class could look like this:

internal class ProcessItemActivity : IPatternActivity<List<object>, List<object>> { private readonly ILogger<ProcessItemActivity> _logger; private readonly IService _service; public ProcessItemActivity( ILogger<ProcessItemActivity> logger, IService service) { _logger = logger; _service = service; } public async Task<List<object>> RunAsync(List<object> input) { _logger.LogInformation("this block of code is executed in parallel batches"); var processedItems = new List<object>(); foreach (var item in input) { processedItems.Add(await _service.ProcessItemAsync(item)); } return processedItems; } }

Configuring Fan Out Fan In

We can configure a FanOutFanIn steop easily by passing options to the method. Here is an example:

[FunctionName("Orchestrator")] 
public async Task RunOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context)
{ 
  await _fluentPatterns 
    .WithContext(context) 
    .RunActivity<GetItemsActivity>() 
    .FanOutFanIn<ProcessItemActivity>(new FanOutFanInOptions( 
      BatchSize: 10, 
      ParallelActivityFunctionsCap: 10)) 
    .RunActivity<ProcessResultsActivity>() 
    .ExecuteAsync

In this example, we are using the FanOutFanIn method and passing options to it. We are specifying the maximum batch size, the maximum number of parallel functions.

What's Next?

Durable Patterns is an open-source project that you can find here. It's also available as a NuGet package.

Our team is planning to add support for the rest of the typical patterns, such as:

• Async HTTP APIs

• Monitoring

• Aggregator (stateful entities).

We will also be adding support for other use cases that are not covered by the official patterns.

We welcome any contributors to join our project and help us improve it!