Title: Contrasting information theoretic decompositions of modulatory and arithmetic interactions in neural information processing systems,
Abstract: Biological and artificial neural systems are composed of many local
processors, and their capabilities depend upon the transfer function that
relates each local processor's outputs to its inputs. This paper uses a recent
advance in the foundations of information theory to study the properties of
local processors that use contextual input to amplify or attenuate transmission
of information about their driving inputs. This advance enables the information
transmitted by processors with two distinct inputs to be decomposed into those
components unique to each input, that shared between the two inputs, and that
which depends on both though it is in neither, i.e. synergy. The decompositions
that we report here show that contextual modulation has information processing
properties that contrast with those of all four simple arithmetic operators,
that it can take various forms, and that the form used in our previous studies
of artificial neural nets composed of local processors with both driving and
contextual inputs is particularly well-suited to provide the distinctive
capabilities of contextual modulation under a wide range of conditions. We
argue that the decompositions reported here could be compared with those
obtained from empirical neurobiological and psychophysical data under
conditions thought to reflect contextual modulation. That would then shed new
light on the underlying processes involved. Finally, we suggest that such
decompositions could aid the design of context-sensitive machine learning
algorithms.