How to get the weights of a linear model by solving normal equation?
Hey I know this is a year old but I wanted to provide the answer in case you haven't got it. So page 124 says how we derive the normal equations, however the following few paragraphs explains how to deal with a bias term.
You've got our dataset right, however we need to add an extra 1 to take the place of our bias in the weight vector. I'll append the ones at the end.
$\mathbf{X} = \begin{bmatrix}
0 &amp; 0 &amp; 1 \\
0 &amp; 1 &amp; 1 \\
1 &amp; 0 &amp; 1 \\
1 &amp; 1 &amp; 1 \\
\end{bmatrix} $
You've got our $y$ vector correct though, i'll write it down below for completeness sakes
$\mathbf{y} = \begin{bmatrix}
0 \\
1 \\
1 \\
0 \\
\end{bmatrix} $
Now we compute solution to the normal equations, $w = (X^{T}X)^{-1}X^{t}y$, below
$\mathbf{w} = \begin{bmatrix}
0 \\
0 \\
\frac{1}{2} \\
\end{bmatrix}$
Now the element in the 3rd position of our vector is the bias term and hence $b = \frac{1}{2}$ and the first two element of $\mathbf{w}$ is our weight vector equalling zero.
Hopefully you find this useful!