Linear Regression of Sine wave using Gradient descent Not working I am writing an algorithm to fit a sine wave. I want to have 4 parameters ( amplitude, frequency, phase &amp; centre position). When I tried to program with all 4 parameters I wasn't able to find a good fit. So I tried to first make it work with each parameter separately and making the other 3 constant hardcoded values. I managed to make this work for all parameters except for the frequency. So I think the problem is the derivative of the error function w.r.t the frequency parameter. As seen below all the other parameters of the sine are hardcoded values except the frequency parameter

%This is iterated for a large number of times
for i = 1:length(x)
%computing hypothesis and error
    TrialY(i) = 4* sin(w2(end) *x(i) + 4) -2;
    Error(end) = Error(end) + (TrialY(i) - y(i))^2;
end
Error(end) = Error(end) / (2*length(x));

dw2 = 0; %derivative value

%derivative of error function with respect to frequency (w2)
for i = 1: length(x)
    dw2 = dw2 + (4* sin(w2(end) *x(i) + 4) -2 - y(i)) * 4 * x(i) * cos(w2(end) * x(i) + 4);
%hardcoded derivative
end
dw2 = 2 * dw2 / length(x); %normalising error

w2(end + 1) = 0;

w2(end) = w2(end - 1) - alpha * dw2; % alpha is the update constant


It seems like the parameter is not converging on the right value with whatever alpha and initial values I give it. While the plot of the Error with time is decreasing but converging on a value > 0.
The problem seems to be that in the case of frequency parameter, the error function is not convex, and has multiple local minima of value greater than 0. The gradient descent is converging to one of these minima. The error will reduce to 0 only if the starting estimate lies in the right convex basin.

For the amplitude and center value parameters, the error functions are convex, and for phase parameter, there are multiple local minima all of value 0.