Async Unit Tests, Part 2: The Right Way

Last time, we looked at incorrect approaches to async unit testing. We also identified the underlying problem: that unit tests do not have an appropriate async context.

At this point, the solution should be pretty obvious: give the unit tests an async context!

It really is that easy! Why, all you have to do is write your own SynchronizationContext implementation. Keep in mind that thread-safety is paramount, because the methods under test may interact with the thread pool or other async contexts. Note that async void methods interact with SynchronizationContext in a different way than other async methods. Oh, and also remember that exceptions need special handling in some cases so their original call stack is preserved appropriately, and if you’re on VS2010 you’ll need to hack this in because there’s no support for it on .NET 4.0.

Just kidding! Ha, ha! The good folks on the Async team have done all the hard work for you. :)

Right Way #1: The Official Approach

If you have the Async CTP installed, then check out the “My Documents\Microsoft Visual Studio Async CTP\Samples(C# Testing) Unit Testing\AsyncTestUtilities” folder. You’ll find not just one, but three async-compatible contexts, ready for you to use!

You should use GeneralThreadAffineContext unless you absolutely need another one. To use it, just copy AsyncTestUtilities.cs, CaptureAndRestorer.cs, and GeneralThreadAffineContext.cs into your test project.

Then, take each unit test and re-write it so that it has a context:

[TestMethod]
public void FourDividedByTwoIsTwo()
{
    GeneralThreadAffineContext.Run(async () =>
    {
        int result = await MyClass.Divide(4, 2);
        Assert.AreEqual(2, result);
    });
}
    
[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void DenominatorIsZeroThrowsDivideByZero()
{
    GeneralThreadAffineContext.Run(async () =>
    {
        await MyClass.Divide(4, 0);
    });
}

Our unit test methods are not async. Each one sets up an async context and passes the actual test into it as an async lambda expression. So, the actual test code can still be written with all the benefits of async/await, and the async context takes care of making sure it runs as expected:

Just as importantly, the async context ensures that tests that should fail, will fail:

[TestMethod]
public void FourDividedByTwoIsThirteen_ShouldFail()
{
    GeneralThreadAffineContext.Run(async () =>
    {
        int result = await MyClass.Divide(4, 2);
        Assert.AreEqual(13, result);
    });
}

And everyone lived happily ever after!

Well, sort of. This solution does work, but it’s a bit cumbersome. Copying code files into each test project? Modifying every unit test to set up its own async context? Really?

Right Way #2: Now with Less Effort!

Boy, if only there was some way to have the MSTest framework apply the async context for us, then we could just write async unit test methods and not worry about it!

Oh yeah - there is. Visual Studio allows you to define a custom “test type.” It really is that easy! Why, all you have to do is… ah, forget it. A custom “async unit test” type is already available:

• Install the AsyncUnitTests-MSTest NuGet package into your test project.
• Add a using Nito.AsyncEx.UnitTests; line.
• Change your [TestClass] attribute to [AsyncTestClass].

Sweet.

Now you can write async unit tests (using async void):

[TestMethod]
public async void FourDividedByTwoIsTwoAsync()
{
    int result = await MyClass.Divide(4, 2);
    Assert.AreEqual(2, result);
}
    
[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public async void DenominatorIsZeroThrowsDivideByZeroAsync()
{
    await MyClass.Divide(4, 0);
}

And it works:

And test failures actually fail:

[TestMethod]
public async void FourDividedByTwoIsThirteenAsync_ShouldFail()
{
    int result = await MyClass.Divide(4, 2);
    Assert.AreEqual(13, result);
}

Sniff… It’s… so… beautiful…

But not quite perfect. You still have to add a NuGet package and remember to change [TestClass] to [AsyncTestClass].

Future Directions

xUnit.NET has recently released first-class support for asynchronous unit tests: in version 1.9 (2012-01-02) and newer, for any test method returning Task/Task<T>, the test framework will wait until the task completes before declaring success/failure. However, as of now, it does not support async void unit tests; this is planned for a future release.

I’ve been in contact with some people inside of Microsoft regarding this issue, and they said they’re aware of it and are considering various options. They wouldn’t give me any details, of course, but they did suggest that I would be “pleasantly surprised” when Visual Studio vNext comes out.

So, that’s where we are today. Hopefully Microsoft will ship built-in async unit test support in Visual Studio vNext, and I’ll be able to look back at this blog post and laugh at how fraught with peril async unit testing used to be.