Linear Regression of Sine wave using Gradient descent Not working
The problem seems to be that in the case of frequency parameter, the error function is not convex, and has multiple local minima of value greater than 0. The gradient descent is converging to one of these minima. The error will reduce to 0 only if the starting estimate lies in the right convex basin.

For the amplitude and center value parameters, the error functions are convex, and for phase parameter, there are multiple local minima all of value 0.