dataproj py from datared import from numpy linalg import svd pinv mu (mu mu reshape ( , mu (mu mu reshape ( , np hstack (mu mu ) , , svd(wwe only need pinv(wr rx ( tt rx ( tt rx ( tt plt plot(rx [, rx [, ,' ',alpha = markersize = plt plot(rx [, rx [, ,' ',alpha = markersize = plt plot(rx [, rx [, ,' ',alpha = markersize = plt show ( logistic regression and softmax classification in example we introduced the logistic (logitregression model as generalized linear model whereconditional on -dimensonal feature vector xthe random response has ber( (xb)distribution with ( /( - the parameter was then learned from the training data by maximizing the likelihood of the training responses orequivalentlyby minimizing the supervised version of the cross-entropy training loss ( ) ln (yi bxi ) = where ( bx /( - and ( bxe- /( - in particularwe have ( bxln xb ( ( bxlog-odds ratio in other wordsthe log-odds ratio is linear function of the feature vector as consequencethe decision boundary { ( bxg( bx)is the hyperplane xb note that typically includes the constant feature if the constant feature is considered separatelythat is [ ]then the boundary is an affine hyperplane in suppose that training on {(xi yi )yields the estimate with the corresponding > learner gt ( /( - the learner can be used as pre-classifier from which we obtain the classifier {gt ( / orequivalentlyb :argmax gt ( ) { , multi-logit in accordance with the fundamental classification rule ( the above classification methodology for the logit model can be generalized to the multi-logit model where the response takes values in the set { the key idea is