What does a conservative technique mean in the context of neural networks?
In this layered graph structure, the
relevance conservation property can be formulated as follows: Let
j and k be indices for neurons of two successive layers. Let Rk be
the relevance of neuron k for the prediction f (x). We define R j←k
as the share of Rk that is redistributed to neuron j in the lower
layer. The conservation property for this neuron imposes
∑j R j←k = Rk.
Likewise, neurons in the lower layer aggregate all relevance coming
from the neurons from the higher layer:
Rj = ∑k R j←k
These two equations, when combined, also ensure a relevance conservation property between layers
Refer to this paper Methods for interpreting and understanding deep neural networks by Grégoire Montavon for more details