Why does over-modelling an adaptive AR NLMS filter fix sharp spikes?
What appears to be happening is, as you start over-modeling, the error signal becomes less and less white.

I modified your code to return the error signal (part of the residue term).

This plot shows the off-zero-lag coefficients of the xcorr of the error for order = 2 (blue), 3 (red), and 4 (green).  As you can see, the close-to-but-not-zero lag terms are getting larger in magnitude. 

If we look at the FFT (spectrum) of the xcorr of the error, then we see that the lower frequency terms (that cause the large drifts) are getting smaller (the error is containing more high frequencies).

So it seems the effect of over-modeling in this case is to high-pass filter the error, which (for this example) is beneficial.





function [th,err]=ar_nlms(y,order,mu)
eps = 0.000000001;
N=length(y);
th=zeros(order,N); // estimated parameters
err = zeros(1,N);
for t=order+1:N
    phi = -y( t-1:-1:t-order, : );
    err(t) = y(t)-phi'*th(:,t-1);
    residue = phi*( err(t) );
    th(:,t) = th(:,t-1) + (mu/(phi'*phi+eps)) * residue;
    size(residue)
end