Title: Empirical Bayes Estimators for High-Dimensional Sparse Vectors,
Abstract: The problem of estimating a high-dimensional sparse vector
$\boldsymbol{\theta} \in \mathbb{R}^n$ from an observation in i.i.d. Gaussian
noise is considered. The performance is measured using squared-error loss. An
empirical Bayes shrinkage estimator, derived using a Bernoulli-Gaussian prior,
is analyzed and compared with the well-known soft-thresholding estimator. We
obtain concentration inequalities for the Stein's unbiased risk estimate and
the loss function of both estimators. The results show that for large $n$, both
the risk estimate and the loss function concentrate on deterministic values
close to the true risk.
Depending on the underlying $\boldsymbol{\theta}$, either the proposed
empirical Bayes (eBayes) estimator or soft-thresholding may have smaller loss.
We consider a hybrid estimator that attempts to pick the better of the
soft-thresholding estimator and the eBayes estimator by comparing their risk
estimates. It is shown that: i) the loss of the hybrid estimator concentrates
on the minimum of the losses of the two competing estimators, and ii) the risk
of the hybrid estimator is within order $\frac{1}{\sqrt{n}}$ of the minimum of
the two risks. Simulation results are provided to support the theoretical
results. Finally, we use the eBayes and hybrid estimators as denoisers in the
approximate message passing (AMP) algorithm for compressed sensing, and show
that their performance is superior to the soft-thresholding denoiser in a wide
range of settings.