title: Streaming data via IEnumerable & TPL Dataflow
body: <p>I am getting items from an upstream API which is quite slow. I try to speed this up by using TPL Dataflow to create multiple connections and bring these together, like this;</p>
  
  <pre class="lang-cs prettyprint-override"><code>class Stuff
  {
      int Id { get; }
  }
  
  async Task&lt;Stuff&gt; GetStuffById(int id) =&gt; throw new NotImplementedException();
  
  async Task&lt;IEnumerable&lt;Stuff&gt;&gt; GetLotsOfStuff(IEnumerable&lt;int&gt; ids)
  {
      var bagOfStuff = new ConcurrentBag&lt;Stuff&gt;();
  
      var options = new ExecutionDataflowBlockOptions
      {
          MaxDegreeOfParallelism = 5
      };
  
      var processor = new ActionBlock&lt;int&gt;(async id =&gt;
      {
          bagOfStuff.Add(await GetStuffById(id));
      }, options);
  
      foreach (int id in ids)
      {
          processor.Post(id);
      }
  
      processor.Complete();
      await processor.Completion;
  
      return bagOfStuff.ToArray();
  }
  </code></pre>
  
  <p>The problem is that I have to wait until I have finished querying the entire collection of <code>Stuff</code> before I can return it to the caller. What I would prefer is that, whenever any of the multiple parallel queries returns an item, I return that item in a <code>yield return</code> fashion. Therefore I don't need to return an <code>sync Task&lt;IEnumerable&lt;Stuff&gt;&gt;</code>, I can just return an <code>IEnumerable&lt;Stuff&gt;</code> and the caller advances the iteration as soon as any items return.</p>
  
  <p>I tried doing it like this;</p>
  
  <pre class="lang-cs prettyprint-override"><code>IEnumerable&lt;Stuff&gt; GetLotsOfStuff(IEnumerable&lt;int&gt; ids)
  {
      var options = new ExecutionDataflowBlockOptions
      {
          MaxDegreeOfParallelism = 5
      };
  
      var processor = new ActionBlock&lt;int&gt;(async id =&gt;
      {
          yield return await GetStuffById(id);
      }, options);
  
      foreach (int id in ids)
      {
          processor.Post(id);
      }
  
      processor.Complete();
      processor.Completion.Wait();
  
      yield break;
  }
  </code></pre>
  
  <p>But I get an error</p>
  
  <blockquote>
    <p>The yield statement cannot be used inside an anonymous method or lambda expression</p>
  </blockquote>
  
  <p>How can I restructure my code?</p>