Title: Stochastic Conjugate Gradient Algorithm with Variance Reduction,
Abstract: Conjugate gradient (CG) methods are a class of important methods for solving
linear equations and nonlinear optimization problems. In this paper, we propose
a new stochastic CG algorithm with variance reduction and we prove its linear
convergence with the Fletcher and Reeves method for strongly convex and smooth
functions. We experimentally demonstrate that the CG with variance reduction
algorithm converges faster than its counterparts for four learning models,
which may be convex, nonconvex or nonsmooth. In addition, its area under the
curve performance on six large-scale data sets is comparable to that of the
LIBLINEAR solver for the L2-regularized L2-loss but with a significant
improvement in computational efficiency