title: TPL Parallel.Foreach with IO and compute intensive tasks
body: <p>I have billions of xml log files on Azure blob storage to be processed, queried and results storage. I am using Parallel.Foreach as processing of files is independent of each other.</p>
  
  <pre><code>Parallel.ForEach&lt;String&gt; (listOfFeatureFiles, file =&gt; { 
    //For each file that was created
    string fileName = file;
    string directoryPath = outputfolderPath + "/" + FeatureFolderName;
    string finalFilePath = directoryPath + "/" + fileName;
  
    DownloadContent();
    XMLParseAndQueryData();
    UploadResultToQueue();
    DeleteLocalCopy();
  });
  </code></pre>
  
  <p>If it were only compute intensive then i might had maximum CPU usage however with my scenario 20% of files are much bigger (in GBs) as compared to rest 80% of files. This usually results in only 50% CPU usage with 4 cores. How can i optimize it to make maximum CPU usage i.e. > 90% ?</p>
  
  <p>My assumption is that once a task is downloading big files, no cpu is used however no new thread is created in the meantime as well which could make use of processing power. I might be wrong about this assumption and will appreciate a concrete link to its negation. </p>