Title: Accelerated Gossip via Stochastic Heavy Ball Method,
Abstract: In this paper we show how the stochastic heavy ball method (SHB) -- a popular
method for solving stochastic convex and non-convex optimization problems
--operates as a randomized gossip algorithm. In particular, we focus on two
special cases of SHB: the Randomized Kaczmarz method with momentum and its
block variant. Building upon a recent framework for the design and analysis of
randomized gossip algorithms, [Loizou Richtarik, 2016] we interpret the
distributed nature of the proposed methods. We present novel protocols for
solving the average consensus problem where in each step all nodes of the
network update their values but only a subset of them exchange their private
values. Numerical experiments on popular wireless sensor networks showing the
benefits of our protocols are also presented.