Two type of calculating R in steepest-descent modeling algorithm
So let's have a look at your calculations:

x1(:,i)=x(N-(i-1):end-(i-1));


Let's say N = 10 and sample = 100 so that end = 110.

For i=1 that means we sum from 10 to 110.

For i=2 that means we sum from 9 to 109.

etc.

Comparing that with:

R(k)=h(k:m)*h(1:m-k+1)';


where m is the same as sample:

For k=1 that means we sum from 1 to 100.

For k=2 that means we sum from 2 to 99.

I think all that's happening is your first calculation is assuming x is zero-padded and the second one is assuming that it's not.

That there are differences between the two is hardly surprising, given that they are summing different (possibly random) values.