▁neural ▁networks . ▁In ▁a ▁distributed ▁representation ▁any ▁given ▁input ▁will ▁tend ▁to ▁create ▁changes ▁in ▁the ▁weights ▁to ▁many ▁of ▁the ▁nodes . ▁Cat ast roph ic ▁forget ting ▁occurs ▁because ▁when ▁many ▁of ▁the ▁weights ▁where ▁" know ledge ▁is ▁stored " ▁are ▁changed , ▁it ▁is ▁impossible ▁for ▁prior ▁knowledge ▁to ▁be ▁kept ▁int act . ▁During ▁sequ ential ▁learning , ▁the ▁inputs ▁become ▁mixed ▁with ▁the ▁new ▁input ▁being ▁super im posed ▁over ▁top ▁of ▁the ▁old ▁input . ▁Another ▁way ▁to ▁concept ual ize ▁this ▁is ▁through ▁visual izing ▁learning ▁as ▁movement ▁through ▁a ▁weight ▁space . ▁This ▁weight ▁space ▁can ▁be ▁li ken ed ▁to ▁a ▁spatial ▁representation ▁of ▁all ▁of ▁the ▁possible ▁combinations ▁of ▁weights ▁that ▁the ▁network ▁can ▁possess . ▁When ▁a ▁network ▁first ▁lear ns ▁to ▁represent ▁a ▁set ▁of ▁patterns , ▁it ▁has ▁found ▁a ▁point ▁in ▁weight ▁space ▁which ▁allows ▁it ▁to ▁recognize ▁all ▁of ▁the ▁patterns ▁that ▁it ▁has ▁seen . ▁However , ▁when ▁the ▁network ▁lear ns ▁a ▁new ▁set ▁of ▁patterns ▁sequ entially ▁it ▁will ▁move ▁to ▁a ▁place ▁in ▁the ▁weight ▁space ▁that ▁allows ▁it ▁to ▁only ▁recognize ▁the ▁new ▁pattern . ▁To ▁recognize ▁both ▁sets ▁of ▁patterns , ▁the ▁network ▁must ▁find ▁a ▁place ▁in ▁weight ▁space ▁that ▁can ▁represent ▁both ▁the ▁new ▁and ▁the ▁old ▁output . ▁One ▁way ▁to ▁do ▁this ▁is ▁by ▁connecting ▁a ▁hidden ▁unit ▁to ▁only ▁a ▁subset ▁of ▁the ▁input ▁units . ▁This ▁reduces ▁the ▁lik elihood ▁that ▁two ▁different ▁inputs