One of the C#'s little hidden gems. If you want to stump your co-workers in C# trivia, than let me introduce you to deferred execution.

So you may or may not have ever heard the term "deferred execution", but I am willing to bet the latter is the most true, for the majority of you. It is also likely that the majority of you use LINQ in your application code, but never really delved deep into how LINQ works below the surface.

Every so often I will come into an interview in which I am asked what deferred execution is, albeit not outright. I am usually given an example on a whiteboard and asked to solve for a specific variable. Their is an intuitive answer that is simple and right in your face, but that is the trick, because that is not the "right" answer. The right answer has to do with deferred execution and when LINQ processes or iterates its queries.

In the simplest of definitions, deferred execution means that the evaluation of an expression is delayed until its realized value is actually required. Deferred execution can greatly improve performance when you have to manipulate large data collections, especially in programs that contain a series of chained queries or manipulations. In the best case, deferred execution enables only a single iteration through the source collection.

The LINQ technologies make extensive use of deferred execution in both the members of core System.Linq classes and in the extension methods in the various LINQ namespaces, such as System.Xml.Linq.Extensions.

Deferred execution is supported directly in the C# language by the yield keyword (in the form of the yield-return statement) when used within an iterator block. Such an iterator must return a collection of type IEnumerator or IEnumerator<T> (or a derived type).

Deferred execution applies to all varieties of LINQ, including LINQ to SQL, LINQ to Objects and LINQ to XML. However, of the three, it is only LINQ to SQL that returns an expression tree by default. Or more specifically, it returns an instance of the IQueryable interface that references an expression tree. A query that returns IEnumerable still supports deferred execution, but at least some larger portion of the result is likely to have been generated than is the case with LINQ to SQL. In other words, all types of LINQ support deferred execution, but LINQ to SQL supports it more fully than LINQ to Objects or LINQ to XML.

This topic shows how deferred execution and lazy evaluation affect the execution of your LINQ to XML queries.


The following example shows the order of execution when using an extension method that uses deferred execution. The example declares an array of three strings. It then iterates through the collection returned by ConvertCollectionToUpperCase.

public static class LocalExtensions
    public static IEnumerable<string>
      ConvertCollectionToUpperCase(this IEnumerable<string> source)
        foreach (string str in source)
            Console.WriteLine("ToUpper: source {0}", str);
            yield return str.ToUpper();

class Program
    static void Main(string[] args)
        string[] stringArray = { "abc", "def", "ghi" };

        var q = from str in stringArray.ConvertCollectionToUpperCase()
                select str;

        foreach (string str in q)
            Console.WriteLine("Main: str {0}", str);

This example produces the following output:
ToUpper: source abc
Main: str ABC
ToUpper: source def
Main: str DEF
ToUpper: source ghi
Main: str GHI

Notice that when iterating through the collection returned by ConvertCollectionToUpperCase, each item is retrieved from the source string array and converted to uppercase before the next item is retrieved from the source string array.

You can see that the entire array of strings is not converted to uppercase before each item in the returned collection is processed in the foreach loop in Main.


At this point, it is important that you understand the semantic difference between the terms "Eager" vs. "Lazy Evaluation" as well, because it applies to this topic.

Eager vs. Lazy Evaluation

When you write a method that implements deferred execution, you also have to decide whether to implement the method using lazy evaluation or eager evaluation.
  • In lazy evaluation, a single element of the source collection is processed during each call to the iterator. This is the typical way in which iterators are implemented.
  • In eager evaluation, the first call to the iterator will result in the entire collection being processed. A temporary copy of the source collection might also be required. For example, the OrderBy method has to sort the entire collection before it returns the first element.
Lazy evaluation usually yields better performance because it distributes overhead processing evenly throughout the evaluation of the collection and minimizes the use of temporary data. Of course, for some operations, there is no other option than to materialize intermediate results.

Have a fun time stumping your co-workers the next time they want to make it seem like they know more than you when it comes to programming in C#!

1 comment: