1 . 0 ) ▁and ▁the ▁activ ations ▁from ▁the ▁next ▁input ▁are ▁( 0 . 0 , ▁ 0 . 9 , ▁ 0 . 1 , ▁ 0 . 9 ) ▁the ▁activation ▁overlap ▁would ▁be ▁( 0 . 0 ▁+ ▁ 0 . 1 ▁+ ▁ 0 . 1 ▁+ ▁ 0 . 9 ▁) ▁/ ▁ 4 ▁= ▁ 0 . 2 7 5 . ▁When ▁using ▁binary ▁representation ▁of ▁input ▁vectors , ▁activation ▁values ▁will ▁be ▁ 0 ▁through ▁ 1 , ▁where ▁ 0 ▁indicates ▁no ▁activation ▁overlap ▁and ▁ 1 ▁indicates ▁full ▁activation ▁overlap . ▁French ▁noted ▁that ▁neural ▁networks ▁which ▁employ ▁very ▁local ized ▁representations ▁do ▁not ▁show ▁cat ast roph ic ▁inter ference ▁because ▁of ▁the ▁lack ▁of ▁overlap ▁at ▁the ▁hidden ▁layer . ▁That ▁is ▁to ▁say , ▁each ▁input ▁pattern ▁will ▁create ▁a ▁hidden ▁layer ▁representation ▁that ▁involves ▁the ▁activation ▁of ▁only ▁one ▁node , ▁so ▁differ ed ▁inputs ▁will ▁have ▁an ▁activation ▁overlap ▁of ▁ 0 . ▁Thus , ▁he ▁suggested ▁that ▁reducing ▁the ▁value ▁of ▁activation ▁overlap ▁at ▁the ▁hidden ▁layer ▁would ▁reduce ▁cat ast roph ic ▁inter ference ▁in ▁distributed ▁networks . ▁Specifically ▁he ▁proposed ▁that ▁this ▁could ▁be ▁done ▁through ▁changing ▁the ▁distributed ▁representations ▁at ▁the ▁hidden ▁layer ▁to ▁' sem i - dist ributed ' ▁representations . ▁A ▁' sem i - dist ributed ' ▁representation ▁has ▁fewer ▁hidden ▁nodes ▁that ▁are ▁active , ▁and / or ▁a ▁lower ▁activation ▁value ▁for ▁these ▁nodes , ▁for ▁each