title: Performance regarding cached thread-safe IEnumerable<T> implementation
body: <p>I created the <code>ThreadSafeCachedEnumerable&lt;T&gt;</code> class intending to increase performance where long running queries where being reused. The idea was to get an enumerator from an <code>IEnumerable&lt;T&gt;</code> and add items to a cache on each call to <code>MoveNext()</code>. The following is my current implementation:</p>
  <pre><code>/// &lt;summary&gt;
  /// Wraps an IEnumerable&amp;lt;T&amp;gt; and provides a thread-safe means of caching the values.&quot;/&gt;
  /// &lt;/summary&gt;
  /// &lt;typeparam name=&quot;T&quot;&gt;&lt;/typeparam&gt;
  class ThreadSafeCachedEnumerable&lt;T&gt; : IEnumerable&lt;T&gt;
  {
      // An enumerator from the original IEnumerable&lt;T&gt;
      private IEnumerator&lt;T&gt; enumerator;
  
      // The items we have already cached (from this.enumerator)
      private IList&lt;T&gt; cachedItems = new List&lt;T&gt;();
  
      public ThreadSafeCachedEnumerable(IEnumerable&lt;T&gt; enumerable)
      {
          this.enumerator = enumerable.GetEnumerator();
      }
  
      public IEnumerator&lt;T&gt; GetEnumerator()
      {
          // The index into the sequence
          int currentIndex = 0;
  
          // We will break with yield break 
          while (true)
          {
              // The currentIndex will never be decremented,
              // so we can check without locking first
              if (currentIndex &lt; this.cachedItems.Count)
              {
                  var current = this.cachedItems[currentIndex];
                  currentIndex += 1;
                  yield return current;
              }
              else
              {
                  // If !(currentIndex &lt; this.cachedItems.Count),
                  // we need to synchronize access to this.enumerator
                  lock (enumerator)
                  {
                      // See if we have more cached items ...
                      if (currentIndex &lt; this.cachedItems.Count)
                      {
                          var current = this.cachedItems[currentIndex];
                          currentIndex += 1;
                          yield return current;
                      }
                      else
                      {
                          // ... otherwise, we'll need to get the next item from this.enumerator.MoveNext()
                          if (this.enumerator.MoveNext())
                          {
                              // capture the current item and cache it, then increment the currentIndex
                              var current = this.enumerator.Current;
                              this.cachedItems.Add(current);
                              currentIndex += 1;
                              yield return current;
                          }
                          else
                          {
                              // We reached the end of the enumerator - we're done
                              yield break;
                          }
                      }
                  }
              }
          }
      }
  
      System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
      {
          return this.GetEnumerator();
      }
  }
  </code></pre>
  <p>I simply <code>lock (this.enumerator)</code> when the no more items appear to be in the cache, just in case another thread is just about to add another item (I assume that calling <code>MoveNext()</code> on this.enumerator from two threads is a bad idea).</p>
  <p>The performance is great when retrieving previously cached items, but it starts to suffer when getting many items for the first time (due to the constant locking). Any suggestions for increasing the performance?</p>
  <hr />
  <p><strong>Edit:</strong> The new <a href="https://docs.microsoft.com/en-us/previous-versions/dotnet/reactive-extensions/hh242985(v=vs.103)" rel="nofollow noreferrer">Reactive Framework</a> solves the problem outlined above, using the <code>System.Linq.EnumerableEx.MemoizeAll()</code> extension method.</p>
  <p>Internally, <code>MemoizeAll()</code> uses a <code>System.Linq.EnumerableEx.MemoizeAllEnumerable&lt;T&gt;</code> (found in the System.Interactive assembly), which is similar to my <code>ThreadSafeCachedEnumerable&lt;T&gt;</code> (sorta).</p>
  <p>Here's an awfully contrived example that prints the contents of an Enumerable (numbers 1-10) very slowly, then quickly prints the contents a second time (because it cached the values):</p>
  <pre><code>// Create an Enumerable&lt;int&gt; containing numbers 1-10, using Thread.Sleep() to simulate work
  var slowEnum = EnumerableEx.Generate(1, currentNum =&gt; (currentNum &lt;= 10), currentNum =&gt; currentNum, previousNum =&gt; { Thread.Sleep(250); return previousNum + 1; });
  
  // This decorates the slow enumerable with one that will cache each value.
  var cachedEnum = slowEnum.MemoizeAll();
  
  // Print the numbers
  foreach (var num in cachedEnum.Repeat(2))
  {
      Console.WriteLine(num);
  }
  </code></pre>