▁and ▁can ▁be ▁applied ▁to ▁both ▁linear ▁and ▁non - linear ▁models . ▁ ▁The ▁total ▁least ▁squares ▁approximation ▁of ▁the ▁data ▁is ▁gener ically ▁equivalent ▁to ▁the ▁best , ▁in ▁the ▁Fro ben ius ▁norm , ▁low - rank ▁approximation ▁of ▁the ▁data ▁matrix . ▁ ▁Linear ▁model ▁ ▁Background ▁ ▁In ▁the ▁least ▁squares ▁method ▁of ▁data ▁model ing , ▁the ▁objective ▁function , ▁S , ▁ ▁is ▁minim ized , ▁where ▁r ▁is ▁the ▁vector ▁of ▁resid uals ▁and ▁W ▁is ▁a ▁weight ing ▁matrix . ▁In ▁linear ▁least ▁squares ▁the ▁model ▁contains ▁equations ▁which ▁are ▁linear ▁in ▁the ▁parameters ▁appearing ▁in ▁the ▁parameter ▁vector ▁, ▁so ▁the ▁resid uals ▁are ▁given ▁by ▁ ▁There ▁are ▁m ▁observations ▁in ▁y ▁and ▁n ▁parameters ▁in ▁ β ▁with ▁m > n . ▁X ▁is ▁a ▁m × n ▁matrix ▁whose ▁elements ▁are ▁either ▁constants ▁or ▁functions ▁of ▁the ▁independent ▁variables , ▁x . ▁The ▁weight ▁matrix ▁W ▁is , ▁ide ally , ▁the ▁inverse ▁of ▁the ▁variance - cov ar iance ▁matrix ▁ ▁of ▁the ▁observations ▁y . ▁The ▁independent ▁variables ▁are ▁assumed ▁to ▁be ▁error - free . ▁The ▁parameter ▁estimates ▁are ▁found ▁by ▁setting ▁the ▁gradient ▁equations ▁to ▁zero , ▁which ▁results ▁in ▁the ▁normal ▁equations ▁ ▁Allow ing ▁observation ▁errors ▁in ▁all ▁variables ▁ ▁Now , ▁suppose ▁that ▁both ▁x ▁and ▁y ▁are ▁observed ▁subject ▁to ▁error , ▁with ▁variance - cov ar iance ▁matrices ▁ ▁and ▁ ▁respectively . ▁In ▁this ▁case ▁the ▁objective ▁function ▁can ▁be