Coefficients of Linear regression for minimizing MSE
For a linear regression we have the loss function

$$J(a,b)=\sum_{n=1}^N(y_n-a-bx_n)^2.$$

The partial derivatives are

$$\dfrac{\partial J}{\partial a}=2\sum_{n=1}^N(y_n-a-bx_n)(-1)$$
$$\dfrac{\partial J}{\partial b}=2\sum_{n=1}^N(y_n-a-bx_n)(-x_n).$$

If we set both derivatives to zero and divide by the sample size $N$ we obtain

$$0=\overline{y}-a-b\overline{x}$$
$$0=\overline{xy}-a\overline{x}-b\overline{x^2}.$$

Now, solve the first equation for $a= \overline{y}-b\overline{x}$ and plug this into the second equation 

$$0=\overline{xy}-\overline{x}\overline{y}+b\overline{x}^2-b\overline{x^2}$$

and solve for $b$ to obtain

$$b=\dfrac{\overline{xy}-\overline{x}\overline{y}}{\overline{x^2}-\overline{x}^2}.$$

The espression in the numerator is the covariance for a sample and the expression in the denominator is the variance of $x$.