title: C# Download data from huge list of urls
body: <p>I have a huge list of web pages which display a status, which i need to check. 
  Some urls are within the same site, another set is located on another site. </p>
  
  <p>Right now i'm trying to do this in a parallel way by using code like below, but i have the feeling that i'm causing too much overhead. </p>
  
  <pre><code>while(ListOfUrls.Count &gt; 0){
    Parallel.ForEach(ListOfUrls, url =&gt;
    {
      WebClient webClient = new WebClient();
      webClient.DownloadString(url);
      ... run my checks here.. 
    });
  
    ListOfUrls = GetNewUrls.....
  }
  </code></pre>
  
  <p>Can this be done with less overhead, and some more control over how many webclients and connections i use/reuse? So, that in the end the job can be done faster? </p>