Introducing NWarpAsync - Emulating yield return using await

When C# 5.0 arrived, it came with a brand new language feature - async functions.
Although asynchronous programming was already possible in previous versions of C#, it was cumbersome. Previous async patterns included the usage of methods such as BeginInvoke or Task.ContinueWith. These patterns were both based on callbacks and they were cumbersome to use.

With C# 5.0, those methods are (mostly) a thing of the past, and async/await is "The Next Big Thing".

I have designed a small library I called NWarpAsync that uses async/await for purposes that are not what it was primarily designed for. It's called "NWarpAsync" because it twists async. It starts with a "N" because, really, what open-source project for .NET doesn't?

Required Knowledge

In order to understand this post, you'll need to understand C#, including yield return and the new async/await features.

Implementing Yield with Await

The first (and so far only) feature of NWarpAsync is emulation of the yield return feature (available since C# 2.0) using await.
I got the idea from watching slides about using yield in node.js to emulate await and I wondered how easy it would be to implement the reverse.

This might not seem very compelling but this is mostly a proof-of-concept anyway and it does nicely support a few cases that C# yield does not.
Here is how it is used:

using System.Linq;
using NWarpAsync.Yield;

class Program
{
    static void Main()
    {
        foreach (int value in new Yielder(async yieldSink => {
            await yieldSink.Yield(1);
            await yieldSink.YieldAll(Enumerable.Range(2, 3));
            await yieldSink.Yield(5);
        }))
        {
            //Prints 1 2 3 4 5 (each number in a different line)
            Console.WriteLine(value);
        }
    }
}

As you see, using await as yield requires the use of a few helper classes/helper methods, but I believe the final result is still reasonably easy to understand.
YieldAll is equivalent to yielding all elements in an enumerable (or doing nothing if the enumerable is null). F# supports this (with the yield! operator), but C# has no short equivalent. Instead, we're stuck with if (x != null) foreach (var value in x) yield return value;

How it is works

The most important class of NWarpAsync.Yield is the Yielder<T> class. A Yielder<T> takes an asynchronous function (the "generator" or "builder" function) and makes it available as a plain old IEnumerable<T>.
The function that Yielder<T> takes in turn receives a parameter of type YieldSink<T>, which exposes the yielding methods. Most of the magic is done in Yielder and YieldSink.

Yielder.GetEnumerator starts the whole process. It creates an enumerator that executes a step in the function Yielder was constructed with every time MoveNext is called. Once the function stops executing (either because it has really returned or because it is awaiting the result of the Yield method), MoveNext checks if the function has yielded any values. If so, then that is value of IEnumerator.Current. If not, then the enumerator has finished, and MoveNext returns false.

YieldSink.GetAwaiter().OnCompleted registers the action that the next MoveNext will execute.

If the enumerator is disposed (e.g. with LINQ Take) before the function has finished executing, then an exception is thrown in the current execution point of the generator function to allow IDisposable resources to be cleaned up properly and finally clauses to execute.

Main Problems and Limitations

It took some effort to make exceptions work properly.
Initially, I made Yielder<T> receive an Action<YieldSink<T>> (as opposed to a function returning a Task). It turns out that unhandled exceptions are nasty for void asynchronous methods.
Switching to Task made this easier.

Unfortunately, the default behavior of Tasks means that the consumer of Yielder will never see "normal" exceptions. Instead, it'll always receive an AggregateException. This could probably be fixed, but I'm not sure it matters. I'll fix it when I'm convinced it's important.

The second problem occurs when the developer writes sink.Yield(value), but forgets to await it. The result is an ugly exception on the next Yield call which simply could not happen with the native yield. In other words, it's easier to get this approach wrong.

Advantages

If await was implemented back in C# 2.0 and there was a new proposal to add yield, I think this library would pretty much kill the arguments in favor of it. Yield can purely be a library feature if await is a language feature. Of course, it's the other way around and yield is here to stay, so that point does not hold.
That said, if language designers want to make a new language that includes await and are wondering if yield matters, I hope this can be helpful.

As I mentioned above, this library supports a F# yield!-like feature that C# does not have.

However, the biggest advantage is that, unlike yield, async/await can be used in lambdas. This means that, for the first time in C#, it is possible to make an anonymous generator function! This is what I did in the example I showed above.
You can even yield enumerations of enumerations using LINQ SelectMany.

Conclusion

Async/Await is a powerful language feature that allows a method to "pause" and then "resume" later.
Features like yield do the same. As such, it's not too hard to implement yield on top of async/await with the help of a proper library.

I'm curious to find out what other "missing" language features can be implemented using async/await. Perhaps some sort of asynchronous collection?

Further Reading

luiscubal/NWarpAsync
The official github repository.

Coroutine - Wikipedia, the free encyclopedia
Coroutines are special functions that can be suspended and resumed. Yield and Async/Await are two coroutine mechanisms.

yield (C# Reference)
The official documentation for yield in C#

await (C# Reference)
The official documentation for await in C#

await anything; - .NET Parallel Programming - Site Home - MSDN Blogs
A blog post explaining how to use await on objects that aren't Tasks.

How yield will transform Node.js
A blog post showing await on top of async in JavaScript.