Often times we have a large list of async Task that are executed against a limited resource. We can use Task.WhenAll for parallelising these unrelated tasks, but it will fire them all off at once, exhausting the resource. So we need a batching mechanism with the same convenience.

Implementation

using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Threading;

namespace Playground
{
    public static class TaskExtensions
    {
        public static async Task WhenAllBatched(IEnumerable<Task> tasks, int maxConcurrency)
        {
            using var semaphore = new SemaphoreSlim(maxConcurrency);

            var pendingTasks = tasks.Select(async task =>
            {
                try
                {
                    semaphore.Wait();
                    await task;
                }
                finally
                {
                    semaphore.Release();
                }
            });

            await Task.WhenAll(pendingTasks);
        }
    }
}

Usage

var tasks = Enumerable.Range(0, 10)
                .Select(laneNo => Task.Run(() => Console.WriteLine($"hello")));

TaskExtensions.WhenAllBatched(tasks, 10)

Note: The execution of the tasks in an IEnumerable<Task> is deferred, i.e. when the collection is created, it is not immediately executed. It is only executed when we iterate over the collection.