Relationship between order of filter and impulse response?
The rate of decay of the impulse response only depends on the distance of the poles of the transfer function to the imaginary axis in the \$s\$-plane, i.e. on the real part of the (generally complex) poles. Remember that for a causal and stable system all poles of the transfer function must be in the left half plane (i.e. the real parts of the poles must be negative), such that the impulse response decays. However, the closer a pole to the imaginary axis the slower does its contribution to the total impulse response decay. The contribution of a complex pole \$s_{\infty}=\sigma+j\omega\$ to the impulse response has the form

$$e^{s_{\infty}t}=e^{\sigma t}e^{j\omega t} $$

which shows that the decay depends on \$\sigma\$, the real part of \$s_{\infty}\$.

This means that in principal even a first order system can have an arbitrarily slow decay of its impulse response. However, for optimal frequency selective filters (Elliptic, Butterworth, etc.) it is the case that higher order filters have poles that are closer to the imaginary axis of the complex \$s\$-plane than low order filters. So the slow decay which you observed for higher order filters is only indirectly related to the filter order. The real reason is the position of the poles close to the \$j\omega\$-axis. These poles result in steeper frequency responses, and sharp transitions in the frequency domain correspond to long (i.e. slowly decaying) impulse responses.