AggregateException handling
I don't think it's a good idea to rethrow only the first/single exception of a particular type. Often, when done right ;-) each exception may contain different data about what went wrong, like entity validation errors may happen for multiple entities or mutliple files may not have been found etc. If you pick only the first one you won't know about the rest and you'll app crashes agian next time.

The Handle and the ThrowIfNonEmpty are fine but Throw is very harmful.

Additionaly I would change the Handle to be Func&lt;TEx, bool&gt;:

public static AggregateException Handle&lt;TEx&gt;(
    this AggregateException source, Func&lt;TEx, bool&gt; handler)
    where TEx : Exception
{
    var exceptions = source.Flatten().InnerExceptions.OfType&lt;TEx&gt;();
    var handlerExceptions = new List&lt;Exception&gt;();
    foreach (var exception in exceptions)
    {
        try
        {
            if (!handler(exception)) 
            {
                handlerExceptions.Add(exception);
            }
        }
        catch (Exception ex)
        {
            handlerExceptions.Add(ex);
        }
    }
    return new AggregateException(source.InnerExceptions
        .Except(exceptions)
        .Concat(handlerExceptions));
}


so that I can for example log it and still be able to say I didn't handle it by returning false.

.Handle((ArgumentException ex) =&gt;
{
   Console.WriteLine(ex.Message);
   return false;
})