title: Parallel.ForEach faster than Task.WaitAll for I/O bound tasks?
body: <p>I have two versions of my program that submit ~3000 HTTP GET requests to a web server.</p>
  
  <p><strong>The first version</strong> is based off of what I read <a href="http://www.michalbialecki.com/2018/04/19/how-to-send-many-requests-in-parallel-in-asp-net-core/" rel="nofollow noreferrer">here</a>. That solution makes sense to me because making web requests is I/O bound work, and the use of async/await along with Task.WhenAll or Task.WaitAll means that you can submit 100 requests all at once and then wait for them all to finish before submitting the next 100 requests so that you don't bog down the web server. I was surprised to see that this version completed all of the work in ~12 minutes - way slower than I expected.</p>
  
  <p><strong>The second version</strong> submits all 3000 HTTP GET requests inside a Parallel.ForEach loop. I use .Result to wait for each request to finish before the rest of the logic within that iteration of the loop can execute. I thought that this would be a far less efficient solution, since using threads to perform tasks in parallel is usually better suited for performing CPU bound work, but I was surprised to see that the this version completed all of the work within ~3 minutes!</p>
  
  <p>My question is why is the Parallel.ForEach version faster? This came as an extra surprise because when I applied the same two techniques against a <strong>different</strong> API/web server, <strong>version 1</strong> of my code was actually faster than version 2 by about 6 minutes - which is what I expected. Could performance of the two different versions have something to do with how the web server handles the traffic? </p>
  
  <p>You can see a simplified version of my code below:</p>
  
  <pre><code>private async Task&lt;ObjectDetails&gt; TryDeserializeResponse(HttpResponseMessage response)
  {
      try
      {
          using (Stream stream = await response.Content.ReadAsStreamAsync())
          using (StreamReader readStream = new StreamReader(stream, Encoding.UTF8))
          using (JsonTextReader jsonTextReader = new JsonTextReader(readStream))
          {
              JsonSerializer serializer = new JsonSerializer();
              ObjectDetails objectDetails = serializer.Deserialize&lt;ObjectDetails&gt;(
                  jsonTextReader);
              return objectDetails;
          }
      }
      catch (Exception e)
      {
          // Log exception
          return null;
      }
  }
  
  private async Task&lt;HttpResponseMessage&gt; TryGetResponse(string urlStr)
  {
      try
      {
          HttpResponseMessage response = await httpClient.GetAsync(urlStr)
              .ConfigureAwait(false);
          if (response.StatusCode != HttpStatusCode.OK)
          {
              throw new WebException("Response code is "
                  + response.StatusCode.ToString() + "... not 200 OK.");
          }
          return response;
      }
      catch (Exception e)
      {
          // Log exception
          return null;
      }
  }
  
  private async Task&lt;ListOfObjects&gt; GetObjectDetailsAsync(string baseUrl, int id)
  {
      string urlStr = baseUrl + @"objects/id/" + id + "/details";
  
      HttpResponseMessage response = await TryGetResponse(urlStr);
  
      ObjectDetails objectDetails = await TryDeserializeResponse(response);
  
      return objectDetails;
  }
  
  // With ~3000 objects to retrieve, this code will create 100 API calls
  // in parallel, wait for all 100 to finish, and then repeat that process
  // ~30 times. In other words, there will be ~30 batches of 100 parallel
  // API calls.
  private Dictionary&lt;int, Task&lt;ObjectDetails&gt;&gt; GetAllObjectDetailsInBatches(
      string baseUrl, Dictionary&lt;int, MyObject&gt; incompleteObjects)
  {
      int batchSize = 100;
      int numberOfBatches = (int)Math.Ceiling(
          (double)incompleteObjects.Count / batchSize);
      Dictionary&lt;int, Task&lt;ObjectDetails&gt;&gt; objectTaskDict
          = new Dictionary&lt;int, Task&lt;ObjectDetails&gt;&gt;(incompleteObjects.Count);
  
      var orderedIncompleteObjects = incompleteObjects.OrderBy(pair =&gt; pair.Key);
  
      for (int i = 0; i &lt; 1; i++)
      {
          var batchOfObjects = orderedIncompleteObjects.Skip(i * batchSize)
              .Take(batchSize);
          var batchObjectsTaskList = batchOfObjects.Select(
              pair =&gt; GetObjectDetailsAsync(baseUrl, pair.Key));
          Task.WaitAll(batchObjectsTaskList.ToArray());
          foreach (var objTask in batchObjectsTaskList)
              objectTaskDict.Add(objTask.Result.id, objTask);
      }
  
      return objectTaskDict;
  }
  
  public void GetObjectsVersion1()
  {
      string baseUrl = @"https://mywebserver.com:/api";
  
      // GetIncompleteObjects is not shown, but it is not relevant to
      // the question
      Dictionary&lt;int, MyObject&gt; incompleteObjects = GetIncompleteObjects();
  
      Dictionary&lt;int, Task&lt;ObjectDetails&gt;&gt; objectTaskDict
          = GetAllObjectDetailsInBatches(baseUrl, incompleteObjects);
  
      foreach (KeyValuePair&lt;int, MyObject&gt; pair in incompleteObjects)
      {
          ObjectDetails objectDetails = objectTaskDict[pair.Key].Result
              .objectDetails;
  
          // Code here that copies fields from objectDetails to pair.Value
          // (the incompleteObject)
  
          AllObjects.Add(pair.Value);
      };
  }
  
  public void GetObjectsVersion2()
  {
      string baseUrl = @"https://mywebserver.com:/api";
  
      // GetIncompleteObjects is not shown, but it is not relevant to
      // the question
      Dictionary&lt;int, MyObject&gt; incompleteObjects = GetIncompleteObjects();
  
      Parallel.ForEach(incompleteHosts, pair =&gt;
      {
          ObjectDetails objectDetails = GetObjectDetailsAsync(
              baseUrl, pair.Key).Result.objectDetails;
  
          // Code here that copies fields from objectDetails to pair.Value
          // (the incompleteObject)
  
          AllObjects.Add(pair.Value);
      });
  }
  </code></pre>