▁can ▁be ▁represented ▁using ▁these ▁types ▁of ▁vectors ▁and ▁some ▁studies ▁report ▁that ▁the ▁degree ▁of ▁inter ference ▁is ▁still ▁problem atic ▁with ▁orthogonal ▁vectors . ▁Simple ▁techniques ▁such ▁as ▁varying ▁the ▁learning ▁rate ▁parameters ▁in ▁the ▁back prop ag ation ▁equation ▁were ▁not ▁successful ▁in ▁reducing ▁inter ference . ▁V ary ing ▁the ▁number ▁of ▁hidden ▁nodes ▁has ▁also ▁been ▁used ▁to ▁try ▁and ▁reduce ▁inter ference . ▁However , ▁the ▁find ings ▁have ▁been ▁mixed , ▁with ▁some ▁studies ▁finding ▁that ▁more ▁hidden ▁units ▁decrease ▁inter ference ▁and ▁other ▁studies ▁finding ▁it ▁does ▁not . ▁ ▁Below ▁are ▁a ▁number ▁of ▁techniques ▁which ▁have ▁empir ical ▁support ▁in ▁successfully ▁reducing ▁cat ast roph ic ▁inter ference ▁in ▁back prop ag ation ▁neural ▁networks : ▁ ▁Node ▁sh ar pen ing ▁technique ▁French ▁( 1 9 9 1 ) ▁ ▁proposed ▁that ▁cat ast roph ic ▁inter ference ▁ar ises ▁in ▁feed forward ▁back prop ag ation ▁networks ▁due ▁to ▁the ▁interaction ▁of ▁node ▁activ ations , ▁or ▁activation ▁overlap , ▁that ▁occur ▁in ▁distributed ▁representations ▁at ▁the ▁hidden ▁layer . ▁Specifically , ▁he ▁defined ▁this ▁activation ▁overlap ▁as ▁the ▁average ▁shared ▁activation ▁over ▁all ▁units ▁in ▁the ▁hidden ▁layer , ▁calculated ▁by ▁sum ming ▁the ▁lowest ▁activation ▁of ▁the ▁nodes ▁at ▁the ▁hidden ▁layer ▁and ▁aver aging ▁this ▁sum . ▁For ▁example , ▁if ▁the ▁activ ations ▁at ▁the ▁hidden ▁layer ▁from ▁one ▁input ▁are ▁( 0 . 3 , ▁ 0 . 1 , ▁ 0 . 9 , ▁