How to derive the sum-of squares error function formula?
There are two interpretations of this formula that I explain one of them. 

\begin{equation} Xw = y \end{equation}

\begin{equation} X^tXw = X^ty \end{equation}

The above is for making sure that you make a square matrix that it has an inverse. It is possible that $X^tX$ does not have any inverse but its chance for linear regression problems is not that much. The reason is that you have a matrix $X$ which belongs to $R^{m \times n}$ which $m$ represents the number of samples and $n$ represents the number of features. Usually, the number of samples is much more than the number of features. Next, 

\begin{equation}(X^tX)^{-1}(X^tX)w = (X^tX)^{-1}X^ty\end{equation}

\begin{equation}w = (X^tX)^{-1}X^ty\end{equation}

Consequently, you have found a closed form for the $w$ linear regression problem which can be generalised to non-linear regression.

Be aware that $(X^tX)^{-1}X^t$ is called the pseudo-inverse of the matrix $X$. The reason is that $X$ is not a square matrix and it does not have inverse but by the mentioned formula you can find its pseudo-inverse. Just multiply it by $X$ and you will get Identity matrix.

There is another interpretation of this. You can find here.