Title: Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression,
Abstract: Concentration inequalities form an essential toolkit in the study of
high-dimensional statistical methods. Most of the relevant statistics
literature is based on the assumptions of sub-Gaussian/sub-exponential random
vectors. In this paper, we bring together various probability inequalities for
sums of independent random variables under much weaker exponential type
(sub-Weibull) tail assumptions. These results extract a part sub-Gaussian tail
behavior in finite samples, matching the asymptotics governed by the central
limit theorem, and are compactly represented in terms of a new Orlicz
quasi-norm - the Generalized Bernstein-Orlicz norm - that typifies such tail
behaviors.
We illustrate the usefulness of these inequalities through the analysis of
four fundamental problems in high-dimensional statistics. In the first two
problems, we study the rate of convergence of the sample covariance matrix in
terms of the maximum elementwise norm and the maximum k-sub-matrix operator
norm which are key quantities of interest in bootstrap procedures and
high-dimensional structured covariance matrix estimation. The third example
concerns the restricted eigenvalue condition, required in high dimensional
linear regression, which we verify for all sub-Weibull random vectors under
only marginal (not joint) tail assumptions on the covariates. To our knowledge,
this is the first unified result obtained in such generality. In the final
example, we consider the Lasso estimator for linear regression and establish
its rate of convergence under much weaker tail assumptions (on the errors as
well as the covariates) than those in the existing literature. The common
feature in all our results is that the convergence rates under most exponential
tails match the usual ones under sub-Gaussian assumptions. Finally, we also
establish a high-dimensional CLT and tail bounds for empirical processes for
sub-Weibulls.