title: Load testing a website
body: <p>I am currently writing a small application to load test a website and am having a few problems.</p>
  
  <pre><code>List&lt;string&gt; pageUrls = new List&lt;string();
  // NOT SHOWN ... populate the pageUrls with thousands of links
  
  var parallelOptions = new System.Threading.Tasks.ParallelOptions();
  parallelOptions.MaxDegreeOfParallelism = 100;
  System.Threading.Tasks.Parallel.ForEach(pageUrls, parallelOptions, pageUrl =&gt;
  {
      var startedOn = DateTime.UtcNow;
  
      var request = System.Net.HttpWebRequest.Create(pageUrl);
  
      var responseTimeBefore = DateTime.UtcNow;
  
      try
      {
          var response = (System.Net.HttpWebResponse)request.GetResponse();
          responseCode = response.StatusCode.ToString();
          response.Close();
      }
      catch (System.Net.WebException ex)
      {
          // NOT SHOWN ... write to the error log
      }
  
      var responseTimeAfter = DateTime.UtcNow;
  
      var responseDuration = responseTimeAfter - responseTimeBefore;
  
      // NOT SHOWN ... write the response duration out to a file
  
      var endedOn = DateTime.UtcNow;
  
      var threadDuration = endedOn - startedOn;
  
      // sleep for one second
      var oneSecond = new TimeSpan(0, 0, 1);
      if (threadDuration &lt; oneSecond)
      {
          System.Threading.Thread.Sleep(oneSecond - threadDuration);
      }
  }
  );
  </code></pre>
  
  <p>When I set the <code>MaxDegreeOfParallelism</code> to a low value such as <code>10</code> everything works fine, the <code>responseDuration</code> stays between 1 and 3 seconds. If I increase the value to <code>100</code> (as in the example) the <code>responseDuration</code> climbs quickly until after around 300 requests the it has reached 25 seconds (and still climbing).</p>
  
  <p>I thought I may be doing something wrong so I also ran <a href="http://jmeter.apache.org/" rel="nofollow">Apache jMeter</a> with the standard <a href="http://jmeter.apache.org/usermanual/build-web-test-plan.html" rel="nofollow">web test plan</a> setup and set the users to 100. After about 300 samples the response times had rocketed to around 40 seconds.</p>
  
  <p>I'm skeptical that my server is reaching its limit. The task manager on the server shows that only 2GB of the 16GB is being used and the processor hangs around 5% effort.</p>
  
  <ol>
  <li>Could I be hitting some limit on the number of simultaneous connections on my client computer? If so, how do I change this?</li>
  <li>Am I forgetting to do something in my code? Clean-up/close connections?</li>
  <li>Could it be that my code is OK and it is in fact my server that just can't handle the traffic?</li>
  </ol>
  
  <p>For reference my client computer that is running the code above is running Windows 7 and is on the same network as the server I am testing. The server is running Windows Server 2008 IIS 7.5 and is a dedicated 8-core 16GB RAM machine.</p>