2 Changes Made Our Azure Durable Functions 3X Faster

For one of our projects, we run a nightly job that generates a batch of files. It’s a long process: pulling records from an upstream system, calling several APIs to gather info about each one, generating a file per record, and writing the results back to storage. For a small batch, this works fine, but as the batch grows, the whole thing takes too long.

We ran a short spike to make it faster. In the end it came down to two small changes: one in the code and one in a config file. Neither did much on its own. Together they took a representative large run from about 710 seconds to about 240 seconds, roughly a 3× speedup. Here’s what they were and why we needed both.

The Sequential Problem

The orchestrator was written the obvious way: loop over the work and await each activity in turn.

foreach (var item in items)
{
    await context.CallActivityAsync(nameof(ProcessItem), item);
}

Nothing wrong with this. It’s correct, it’s easy to read, and for a small batch it’s plenty fast. The problem is that it processes one item at a time, start to finish, no matter how many cores or workers are sitting idle. As the batch grew, the runtime grew right along with it, one item after another.

Change 1: Fan out the work.

Durable Functions solves this with fan-out/fan-in. Instead of awaiting each activity in sequence, you kick them all off and await them together.

var tasks = items.Select(item =>
    context.CallActivityAsync(nameof(ProcessItem), item));

await Task.WhenAll(tasks);

Conceptually, this is the whole win. Instead of a single line of work stretching across the entire run, the same items overlap and finish far sooner.

In theory, this should fix the issue, right? We’re now running everything at the same time. Not quite, and the reason is easy to miss.

Change 2: Raise the concurrency limit.

Our function host was configured to run one activity at a time. In host.json:

{
  "extensions": {
    "durableTask": {
      "maxConcurrentActivityFunctions": 8
    }
  }
}

That value by default had been pinned at 1. And here’s the thing: with the host allowed to run only one activity at a time, it does not matter how you write the orchestrator. Task.WhenAll will happily schedule all the work, and then the host will run it one activity at a time anyway. The fan-out code expressed parallelism the runtime couldn’t act on.

Raise the limit (we raised it to 8 as shown above), and the fan-out finally has somewhere to go. This is the honest crux of the whole exercise: the code change and the config change are a matched pair. The code expresses the parallelism; the config permits it. Miss either one and nothing gets faster. If you’ve ever added Task.WhenAll to a Durable orchestrator and seen no improvement, this is the first place to look.

One caveat worth stating: that concurrency limit isn’t only a throttle on your function. It’s protecting whatever your activities call. Ours call several external APIs, so we couldn’t just crank the limit up without overloading them. We raised it to a number those dependencies can absorb, informed by what we saw them tolerate under load.

With both changes in place:

Why 3x and not 8x?

We raised the limit well past 3, so why only ~3x? Because not all of the run is parallelizable. Amdahl’s law puts a ceiling on any speedup: speedup = 1 / ((1 - p) + p/N), where p is the fraction of the work that can run in parallel and N is how many run at once. The catch is the (1 - p) term. Whatever part is stuck being serial never goes away, so pushing N higher gives smaller and smaller returns.

So your mileage will vary depending on how your functions are set up. The important part is that these two changes will still speed things up, even if your number isn’t 3×.

Takeaways

• Durable fan-out needs two things: the code to fan out (Task.WhenAll) and the config to allow it (maxConcurrentActivityFunctions). Miss either and nothing changes.
• The concurrency limit protects your downstream dependencies, not just your function. Raise it to a number they can handle, not the highest number that works.
• Don’t expect the speedup to match your concurrency number. Whatever part of the run stays sequential sets the ceiling, which is why we got 3× and not 8×.

Two changes, 3x faster. The real lesson was that Durable fan-out is a two-part switch that only works when you flip both halves.