Async-Await in .NET is not for performance – it’s for readability

While I see the LINQ on the .NET platform one of the best if not the best language feature, I think the async-await idea is more like smoke in the mirror than anything else. Actually, it’s even worse: as the underlying operations are not trivial in some cases we are better off using linear code.

Idea based on desktop apps

The whole async-await idea is mostly coming from the cumbersomeness of desktop GUI applications and UI threads: only a single thread can modify the UI so when a long running process finishes we need to ask the UI thread to display the result for us but the operation itself cannot update the UI. It’s not a language problem; it’s a Windows/framework problem. However, Microsoft decided to “solve” it with a language feature, because:

• Concurrent programming is a hot topic and Microsoft wanted to “be there”
• C# language evolves fast so adding a new feature is quite easy

While the latter is a great thing (hey Java, can you hear me?), adding a language feature because of marketing reasons is just plain wrong.

Myth #1: Await will make the code run faster

Aysnc-await is a unique language feature because it’s implemented more as code generator than anything else. Microsoft decided to support the async-await keywords with generated classes and a huge, somewhat undocumented framework that just “solves your problems”. The underlying implementation is so hidden, we tend to believe that it will just make the code faster somehow.

It will not. It will reorganize your code so you don’t have to write callbacks to complete the rest of the function when a long running process completes. The rewriting of your linear code to emulate the callbacks is massive, it worth to have a look at it using ILSpy. Anyhow, the Task<> that is created by an async function was there before, nothing new about it: it will be scheduled to run on the ThreadPool; which, again, was there from 1.0. However, in certain cases (like in a loop) the execution will be essentially sequential. No parallelism, no speedup.

On the other hand, while it can be really helpful in a desktop application it makes little or no sense at all in a server environment – like ASP.NET. Unfortunately VS2013 will generate sample ASP.NET project with async-awaits all over. It is wrong! Why?

• Servers are optimized to handle lot of parallel requests – they will manage the threads for you, no need for extra "optimizations"
• A single request should never take too long – use queues if that’s the case and refresh the client using Javascript instead

Unfortunately Microsoft decided to return Task<>  in the new ASP.NET api calls forcing us to either call .Wait() or to use async-await. As typical HTTP is around 100-300ms the whole idea of asynchronously serving all requests in a synchronous environment is somewhat flawed. Remember, the user is actually waiting for your HTTP response.

Myth #2: Creating a thread is expensive

The biggest argument against creating a new thread for your long running task is that thread creation is expensive. But how expensive exactly? I did two quick measurements:

• Start 1000 threads and wait (Thread.Sleep) in all for 1 second – all threads completed in 1136ms, meaning that the overhead for creating, starting, and stopping a single thread was around 0.13ms
• Start 1000 threads and do not do any work in them – all threads completed in 95ms, meaning that the overhead was 0.095ms per thread

Let’s say 0.1ms per thread is the overhead and creating even 1000 threads is not too stressing for any modern machines – including my average spec laptop. Let’s say you create a brand new thread for every request (bad idea) and a request takes 100ms to complete. That’s 0.1% overhead. It wouldn’t really rush to optimize this overhead.

However, if you have more than 1000 requests per sec (which is more than 86 million requests per day) you really should use some kind of queuing to limit the number of concurrent threads – but all server environments like ASP.NET does that for you, don’t worry.

Myth #3: Concurrent programming can be easy

In an average environment concurrent programming is pretty much unnecessary – no real need to squeeze every computing bit out from a CPU so a standard sequential program will just do fine. For the rare cases when we need to update the UI of a desktop app or do a little background processing, async-await will be more than enough.

However, for any high performance computing projects forget about the features offered by any language or framework. You will need to understand work scheduling, synchronization, hardware bottlenecks, data layout patterns in memory, and in some cases even SIMD instructions and branch prediction. No IDE, programming language or framework will understand that for you. Even in concurrent languages like Google Go you have to understand synchronization quite well to efficiently use your resources.

As currently we operate with a few CPU cores on a large, common, shared memory, concurrent programming is far from trivial – my guess is that until we change our hardware architecture it always will be.

Measurements

Out of curiosity I measured how long it would take to calculate the square root of a large amount of numbers (1000 times of 0, 1 million, 3 million numbers) divided into 1000 work units using different approaches (source codes below):

• Single thread – do all calculations on a single thread
• Lot of threads – try to start 1000 threads to do it (actually around 8 can start because the CPU is too busy running other calculations. This is a surprisingly good self-limiting side effect).
• Lot of async – async done in a loop (which will essentially run sequentially)
• Parallel.For – using the built in .NET parallel library
• Task WhenAll – using the WhenAll() synchronizer and simply creating an running the tasks
• ThreadPool – Using the default QueueUserWorkItem on the default thread pool

Couple things to take away from the chart:

• Parallel.For is slightly faster than other solutions
• Misusing async-await will yield linear performance without compiler warning
• Creating a lot of threads to do the work is only slightly slower than the best performing solution (4% slower in this case).

So is async-await bad?

I think it would be crucial for everyone to understand the internals of async-await before jumping in because really surprising side effects can and will pop up - there are just too many questions around weird behavior of async-await on Stackoverflow. However, the idea is very useful in situations where a callback code would severely undermine the readability of the code, especially on desktop apps. Anywhere else? Not too sure.

Code snippets

private static void SingleThread()
{
    Stopwatch sw = Stopwatch.StartNew();
    for (int i = 0; i < n; i++)
    {
        Sq();
    }
    Console.WriteLine(sw.ElapsedMilliseconds);
}

public static void ParallelForeach()
{
    var sw = Stopwatch.StartNew();

    Parallel.For(0, n, i =>
    {
        Sq();
    });

    Console.WriteLine(n + " parallels foreach finished in " + sw.ElapsedMilliseconds);
}

public static async void LotofAsync()
{
    var sw = Stopwatch.StartNew();

    for (int i = 0; i < n; i++)
    {
        await Task.Factory.StartNew(
        () =>
        {
            Sq();
        });
    }

    Console.WriteLine(n + " awaits started and finished in " + sw.ElapsedMilliseconds);
}

public static void TaskWhenAll()
{
    var sw = Stopwatch.StartNew();
    var tasks = new List<Task>();

    for (int i = 0; i < n; i++)
    {
        var t = Task.Factory.StartNew(
        () =>
        {
            Sq();
        });
        tasks.Add(t);
    }

    Task.WhenAll(tasks).Wait();

    Console.WriteLine(n + " tasks WhenAll finished in " + sw.ElapsedMilliseconds);
}

public static void Threadpool()
{
    var sw = Stopwatch.StartNew();
    var tasks = new List<Task>();

    int total = n;
    var re = new ManualResetEvent(false);

    for (int i = 0; i < n; i++)
    {
        ThreadPool.QueueUserWorkItem(new WaitCallback(o =>
        {
            Sq();
            Interlocked.Decrement(ref total);
            if (total == 0)
            {
                re.Set(); // ThreadPool does not support waiting on tasks by itself.
            }
        }));
    }

    re.WaitOne();

    Console.WriteLine(n + " tasks WhenAll finished in " + sw.ElapsedMilliseconds);
}

public static void LotofThreads()
{
    var sw = Stopwatch.StartNew();

    var threads = new List<Thread>();

    for (int i = 0; i < n; i++)
    {
        var t = new Thread(new ThreadStart(() =>
        {
            Sq();
        }));

        threads.Add(t);
        t.Start();
    }

    threads.ForEach(t => t.Join());

    Console.WriteLine(n + " threads started and finished in " + sw.ElapsedMilliseconds);
}

private static double Sq()
{
    double sum = 0;
    for (int j = 0; j < 1 * 1000 * 1000; j++)
    {
        sum += Math.Sqrt(j);
    }
    return sum;
}