WhenAll for .NET 3.5
int count = remainingTasks.Count();


Since remainingTasks is a List, you can use the Count property here.



var exceptions = new List&lt;Exception&gt;();


This variable should be declared where it's used: in the if Interlocked.Decrement block.



if (task1.IsFaulted)


This means you're treating canceled tasks the same as successfully completed ones. Instead, you can check whether Exception is null.

You could also rewrite the inner foreach using LINQ:

var exceptions = remainingTasks
    .Select(task =&gt; task.Exception)
    .Where(e =&gt; e != null)
    .ToList();