Matrix Dimension for Linear regression coefficients While reading about least squares implementation for machine learning I came across this passage in the following two photos:



Perhaps I’m misinterpreting the meaning of beta  but if X^T has dimension 1 x p and beta has dimension p x K, then hat{Y}  would have dimension 1 x K and would be a row vector. According to the text, vectors are assumed column vectors unless otherwise noted. 

Can someone provide clarification?

Edit: the matrix notation in this text confuses me. The pages preceding the above passage read:




Should the matrix referenced not have dimensions p x N, assuming a p-vector is a column vector with p elements?

Note: The passage is taken from “Elements of Statistical Learning” by Hastie, Tibshirani, &amp; Friedman.
I have used W as beta and Y_pred as Y_hat.

Apparently, as far as conventions go both feature vector X and weight vector W are assumed as column vectors. Although this is not important in your case but when we use Neural Nets, this is particularly important, and a weight vector for a layer is given by p*N_n where p is the number of features being input from the previous layer and N_n is the number of nodes in the layer. Check a NN structure and you will understand what I am saying.

As far as the special case in your current question, the book is correct, although comments say Y_predicted should be vector, it is not, since we are doing dot product between X and W (although it is represented in a vectorised form).

Also  how can W  have dimensions p*K (assuming it is not a NN), it means multiple solutions to the same problem? Instead it is X which will have dimensions p*K and W will have p*1. It means you have K training examples with p features. 

If its a NN with K nodes then dimensions for X like p*N and W as p*K is totally possible. The only downside of this notation is that you have to again take a transpose of the result.

Here is the explanation of the notation used:

Notation for NN