▁only ▁difference ▁is ▁in ▁how ▁we ▁interpret ▁this ▁result . ▁▁ ▁Pro jection ▁▁ ▁For ▁mathemat icians , ▁O LS ▁is ▁an ▁approximate ▁solution ▁to ▁an ▁over det erm ined ▁system ▁of ▁linear ▁equations ▁, ▁where ▁ β ▁is ▁the ▁unknown . ▁Assuming ▁the ▁system ▁cannot ▁be ▁solved ▁exactly ▁( the ▁number ▁of ▁equations ▁n ▁is ▁much ▁larger ▁than ▁the ▁number ▁of ▁unknown s ▁p ), ▁we ▁are ▁looking ▁for ▁a ▁solution ▁that ▁could ▁provide ▁the ▁smallest ▁dis cre p ancy ▁between ▁the ▁right - ▁and ▁left - ▁hand ▁sides . ▁In ▁other ▁words , ▁we ▁are ▁looking ▁for ▁the ▁solution ▁that ▁satisfies ▁▁ ▁where ▁|| · || ▁is ▁the ▁standard ▁L 2   norm ▁in ▁the ▁n - dimensional ▁E uclidean ▁space ▁R n . ▁The ▁predicted ▁quantity ▁X β ▁is ▁just ▁a ▁certain ▁linear ▁combination ▁of ▁the ▁vectors ▁of ▁re gress ors . ▁Thus , ▁the ▁resid ual ▁vector ▁ ▁will ▁have ▁the ▁smallest ▁length ▁when ▁y ▁is ▁project ed ▁orth og on ally ▁onto ▁the ▁linear ▁sub space ▁sp anned ▁by ▁the ▁columns ▁of ▁X . ▁The ▁O LS ▁estim ator ▁ ▁in ▁this ▁case ▁can ▁be ▁interpreted ▁as ▁the ▁coefficients ▁of ▁vector ▁decomposition ▁of ▁ ▁along ▁the ▁basis ▁of ▁X . ▁ ▁In ▁other ▁words , ▁the ▁gradient ▁equations ▁at ▁the ▁minimum ▁can ▁be ▁written ▁as : ▁ ▁A ▁geomet rical ▁interpretation ▁of ▁these ▁equations ▁is ▁that ▁the ▁vector ▁of ▁resid uals , ▁ ▁is ▁orthogonal ▁to ▁the ▁column ▁space ▁of ▁X , ▁since ▁the ▁dot ▁product ▁ ▁is