Title: Estimator of Prediction Error Based on Approximate Message Passing for Penalized Linear Regression,
Abstract: We propose an estimator of prediction error using an approximate message
passing (AMP) algorithm that can be applied to a broad range of sparse
penalties. Following Stein's lemma, the estimator of the generalized degrees of
freedom, which is a key quantity for the construction of the estimator of the
prediction error, is calculated at the AMP fixed point. The resulting form of
the AMP-based estimator does not depend on the penalty function, and its value
can be further improved by considering the correlation between predictors. The
proposed estimator is asymptotically unbiased when the components of the
predictors and response variables are independently generated according to a
Gaussian distribution. We examine the behaviour of the estimator for real data
under nonconvex sparse penalties, where Akaike's information criterion does not
correspond to an unbiased estimator of the prediction error. The model selected
by the proposed estimator is close to that which minimizes the true prediction
error.