Why does gradient descent gives me much better Relative Squared Error then the Least Squares approach?
The reason is that the two metrics, mean absolute percentage error (MAPE) and mean square error (MSE) are optimising to different targets. Improving one can be done at the expense of the other.

As a simple example, consider this data:

x = [ 0,  1,  2,  3,  4,  5]
y = [ 3,  5, 10, 10, 11, 15]


The best fit mean squared error (MSE) for a line on this data is $\hat{y} = 2.23x + 3.43$, which has MSE of $1.18$, and a mean absolute percentage error (MAPE) of $11.0$%.

The best fit mean absolute percentage loss for a line on this data is $\hat{y} = 2.35x + 2.99$, which has a MSE of $1.24$, and a MAPE of $8.34$%. 

You can see that optimising for MAPE gives a worse MSE, and vice-versa.

The difference can get extreme when there is a large range for y values (in terms of orders of magnitude covered), because optimising for MAPE will favour being more accurate on small values at the expense of larger ones. So if we change y to be:

y = [ 1,  2, 10, 10, 11, 20]


Then optimising mean abs percentage gives the line $\hat{y} = 3.78x + 1.05$ with MSE $7.09$ and MAPE $21.9$%. But optimising for mean square error gives $\hat{y} = 3.49x + 0.286$ with MSE  $4.56$ and MAPE $62.9$% - this is a larger difference, and I suspect that your data has a large range of target variable causing a similar effect.



You can potentially get closer results using the Least Squares regressor by using a transformed target variable $z = log(y)$ and transforming back at the end. This still won't be quite the same, but it does reduce the difference significantly - in my last example if I try this, I get MAPE $24.2$% - compared to $21.9$% for optimising MAPE directly.