Title: Provable Smoothness Guarantees for Black-Box Variational Inference,
Abstract: Black-box variational inference tries to approximate a complex target
distribution though a gradient-based optimization of the parameters of a
simpler distribution. Provable convergence guarantees require structural
properties of the objective. This paper shows that for location-scale family
approximations, if the target is M-Lipschitz smooth, then so is the objective,
if the entropy is excluded. The key proof idea is to describe gradients in a
certain inner-product space, thus permitting use of Bessel's inequality. This
result gives insight into how to parameterize distributions, gives bounds the
location of the optimal parameters, and is a key ingredient for convergence
guarantees.