Title: Exact Diffusion for Distributed Optimization and Learning --- Part II: Convergence Analysis,
Abstract: Part I of this work [2] developed the exact diffusion algorithm to remove the
bias that is characteristic of distributed solutions for deterministic
optimization problems. The algorithm was shown to be applicable to a larger set
of combination policies than earlier approaches in the literature. In
particular, the combination matrices are not required to be doubly stochastic,
which impose stringent conditions on the graph topology and communications
protocol. In this Part II, we examine the convergence and stability properties
of exact diffusion in some detail and establish its linear convergence rate. We
also show that it has a wider stability range than the EXTRA consensus
solution, meaning that it is stable for a wider range of step-sizes and can,
therefore, attain faster convergence rates. Analytical examples and numerical
simulations illustrate the theoretical findings.