title: Parallel Linq query optimization
body: <p>For some time now I've been structuring my code around methods with no side-effects in order to use parallel linq to speed things up. Along the way I've more than once stumbled on lazy evaluation making things worse instead of better and I would like to know if there are any tools to help with optimizing parallel linq queries. </p>
  
  <p>I ask because I recently refactored some embarrassingly parallel code by modifying some methods and peppering <code>AsParallel</code> in certain key places. The run time went down from 2 minutes to 45 seconds but it was clear from the performance monitor that there were some places where all the cores on the CPU were not being fully utilized. After a few false starts I forced some of the queries to execute by using <code>ToArray</code> and the run time went down even further to 16 seconds. It felt good to reduce the run time of the code but it was also slightly disconcerting because it was not clear where in the code queries needed to be forced with <code>ToArray</code>. Waiting until the last minute for the query to execute was not the optimal strategy but it was not clear at all at what points in the code some of the subqueries needed to be forced in order to utilize all the CPU cores.</p>
  
  <p>As it is I have no idea how to properly pepper <code>ToArray</code> or other methods that force linq computations to execute in order to gain maximum CPU utilization. So are there any general guidelines and tools for optimizing parallel linq queries?</p>
  
  <p>Here's a pseudo-code sample:</p>
  
  <pre><code>var firstQuery = someDictionary.SelectMany(FirstTransformation);
  var secondQuery = firstQuery.Select(SecondTransformation);
  var thirdQuery = secondQuery.Select(ThirdTransformation).Where(SomeConditionCheck);
  var finalQuery = thirdQuery.Select(FinalTransformation).Where(x =&gt; x != null);
  </code></pre>
  
  <p><code>FirstTransformation</code>, <code>SecondTransformation</code>, <code>ThirdTransformation</code> are all CPU bound and in terms of complexity they are a few 3x3 matrix multiplications and some <code>if</code> branches. <code>SomeConditionCheck</code> is pretty much a <code>null</code> check. <code>FinalTransformation</code> is the most CPU intensive part of the code because it will perform a whole bunch of line-plane intersections and will check polygon containment for those intersections and then extract the intersection that is closest to a certain point on the line.</p>
  
  <p>I have no idea why the places where I put <code>AsParallel</code> reduced the run time of the code as much as it did. I have now reached a local minimum in terms of run time but I have no idea why. It was just dumb luck that I stumbled on it. In case you're wondering the places to put <code>AsParallel</code> are the first and last lines. Putting <code>AsParallel</code> anywhere else will only increase the run time, sometimes by up to 20 seconds. There is also a hidden <code>ToArray</code> hiding in there on the first line.</p>