Closures in C#, necessary or not?
I would recommend reading this article by Eric Lippert 

You'll definitally want an extension method for this... let me hit the google real quick.... Using some inspired code from this SO post I wrote this.

static class Extensions
{
    private const HashSet&lt;Type&gt; NumericTypes = new HashSet&lt;Type&gt; 
        {typeof(decimal), typeof(byte), typeof(sbyte), typeof(short), typeof(ushort),
        typeof(int),typeof(uint), typeof(long), typeof(ulong), typeof(float), typeof(double)};

    /// &lt;summary&gt;Determines whether the specified object is numeric.&lt;/summary&gt;
    public static bool IsNumeric(this object obj) { return NumericTypes.Contains(obj.GetType()); }
}


Now in your code you can just say

if(!row[columnName].IsNumeric())
    throw new Exception();


However, now I will review your code in general.


You should not throw Exception, you should always be specific with your exception throwing. In this case I probably would throw an ArgumentException, but really this seems like a specific case where you should define your own.
I'm a bit confused by your declaring isNumeric outside the loop, when you don't use it outside that scope.
instead of saying if (isNumeric == false) just say if(!isNumeric)
I don't see why you nested this function inside your loop. its not taking full advantage of the fact that it is nested, and could just aswell be a function outside. Plus why declare it if you might be throwing an exception.. might as well wait. And while you're waiting, lets just take this one liner and make it a statement instead. (My thoughts jumped around there a bit)


In the end your code could just look like.

public static double GetColumnAverage(DataTable dt, string columnName)
{
    foreach (DataRow row in dt.Rows)
        if (!row[columnName].IsNumeric())
            throw new ArgumentException("-Something meaningful here-");
    return dt.AsEnumerable().Where(x =&gt; x[columnName] != DBNull.Value).Average(x =&gt; x.Field&lt;dynamic&gt;(columnName));
}