Title: Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference,
Abstract: We propose a simple and general variant of the standard reparameterized
gradient estimator for the variational evidence lower bound. Specifically, we
remove a part of the total derivative with respect to the variational
parameters that corresponds to the score function. Removing this term produces
an unbiased gradient estimator whose variance approaches zero as the
approximate posterior approaches the exact posterior. We analyze the behavior
of this gradient estimator theoretically and empirically, and generalize it to
more complex variational distributions such as mixtures and importance-weighted
posteriors.