A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanism to accelerate the mixing of Langevin diffusion. Our algorithm is motivated by its mean field partial differential equation (PDE), which is a Fokker-Planck equation supplemented by a nonlocal birth-death term. This PDE can be viewed as a gradient flow of the Kullback-Leibler divergence with respect to the Wasserstein-Fisher-Rao metric. We prove that under some assumptions the asymptotic convergence rate of the nonlocal PDE is independent of the potential barrier, in contrast to the exponential dependence in the case of the Langevin diffusion. We illustrate the efficiency of the birth-death accelerated Langevin method through several analytical examples and numerical experiments.
@cite_43: We study an interacting particle system in @math motivated by Stein variational gradient descent [Q. Liu and D. Wang, Proceedings of NIPS, 2016], a deterministic algorithm for approximating a given probability density with unknown normalization based on particles. We prove that in the large particle limit the empirical measure of the particle system converges to a solution of a nonlocal and nonlinear PDE. We also prove the global existence, uniqueness, and regularity of the solution to the limiting PDE. Finally, we prove that the solution to the PDE converges to the unique invariant solution in a long time limit.
@cite_42: We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein’s identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.
@cite_7: Stein variational gradient descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate given distributions, based on a gradient-based update constructed to optimally decrease the KL divergence within a function space. This paper develops the first theoretical analysis on SVGD. We establish that the empirical measures of the SVGD samples weakly converge to the target distribution, and show that the asymptotic behavior of SVGD is characterized by a nonlinear Fokker-Planck equation known as Vlasov equation in physics. We develop a geometric perspective that views SVGD as a gradient flow of the KL divergence functional under a new metric structure on the space of distributions induced by Stein operator.
The proposed sampling scheme involves interacting particles that undergo Langevin diffusion and birth-death process. Other sampling schemes via interacting particles have been proposed recently, including the Stein variational gradient descent (SVGD) flow @cite_42 @cite_7 (see also its continuous limit studied in @cite_43 ). Unlike the samplers based on Stein discrepancy, which replaces the random noise in Langevin dynamics by repulsion of particles, the sampling scheme proposed in this work employs birth-death process to enhance the mixing of existing sampling schemes. In fact, it can also be potentially combined with SVGD to improve its convergence.