Title: Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks,
Abstract: The quest for biologically plausible deep learning is driven, not just by the
desire to explain experimentally-observed properties of biological neural
networks, but also by the hope of discovering more efficient methods for
training artificial networks. In this paper, we propose a new algorithm named
Variational Probably Flow (VPF), an extension of minimum probability flow for
training binary Deep Boltzmann Machines (DBMs). We show that weight updates in
VPF are local, depending only on the states and firing rates of the adjacent
neurons. Unlike contrastive divergence, there is no need for Gibbs
confabulations; and unlike backpropagation, alternating feedforward and
feedback phases are not required. Moreover, the learning algorithm is effective
for training DBMs with intra-layer connections between the hidden nodes.
Experiments with MNIST and Fashion MNIST demonstrate that VPF learns reasonable
features quickly, reconstructs corrupted images more accurately, and generates
samples with a high estimated log-likelihood. Lastly, we note that,
interestingly, if an asymmetric version of VPF exists, the weight updates
directly explain experimental results in Spike-Timing-Dependent Plasticity
(STDP).