Nitrogen inversion barriers of trimethylamine and N,N-diisopropylmethylamine
So I thought about this for a bit and I think I have an answer. Be warned, I might be wrong.

The inversion process involves a conversion from $\mathrm{sp^2}$ to $\mathrm{sp^3}$. So $\mathrm{sp^2}$ hybridization increases angle between these bulky groups, which would be in fact favorable. Hence the low activation energy.