Before reading this page, please check out the Linear Curve Fitting page. Many of the principles mentioned there will be re-used here, and will not be explained in as much detail.
A graph showing how different degree polynomials can be fitted to data.
How do we find the minimum of this error function? We use the derivative. If we can differentiate \(err\), we have an equation for the slope. We know that the slope will be 0 when the error is at a minimum.
See https://github.com/gbmhunter/BlogAssets/tree/master/Mathematics/CurveFitting/polynomial for Python code which performs these calculations.
Four data points and a 2 degree polynomial of best fit (using the least squares method).