Title: Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization,
Abstract: Recent studies on diffusion-based sampling methods have shown that Langevin
Monte Carlo (LMC) algorithms can be beneficial for non-convex optimization, and
rigorous theoretical guarantees have been proven for both asymptotic and
finite-time regimes. Algorithmically, LMC-based algorithms resemble the
well-known gradient descent (GD) algorithm, where the GD recursion is perturbed
by an additive Gaussian noise whose variance has a particular form. Fractional
Langevin Monte Carlo (FLMC) is a recently proposed extension of LMC, where the
Gaussian noise is replaced by a heavy-tailed {\alpha}-stable noise. As opposed
to its Gaussian counterpart, these heavy-tailed perturbations can incur large
jumps and it has been empirically demonstrated that the choice of
{\alpha}-stable noise can provide several advantages in modern machine learning
problems, both in optimization and sampling contexts. However, as opposed to
LMC, only asymptotic convergence properties of FLMC have been yet established.
In this study, we analyze the non-asymptotic behavior of FLMC for non-convex
optimization and prove finite-time bounds for its expected suboptimality. Our
results show that the weak-error of FLMC increases faster than LMC, which
suggests using smaller step-sizes in FLMC. We finally extend our results to the
case where the exact gradients are replaced by stochastic gradients and show
that similar results hold in this setting as well.