How does "linear algebraic" weight training function work?
Your goal is to find a $w$ such that

$$Xw \approx y$$

and way to model this problem is to minimize the objective function:

$$\min_w\|Xw-y\|^2.$$

Differentiating with respect to $w$ and equate it to zero gives us

$$2X^T(Xw-y)=0$$

$$X^TXw-X^Ty=0$$

$$X^TXw=X^Ty$$

if $X^TX$ is invertible, then we have 

$$w=(X^TX)^{-1}(X^Ty)$$

Remark:

We tend to avoid computing inverse and prefer gradient-based method. Complexity of the normal equation method is cubic.