Seemingly unrelated regression is a natural framework for regressing multiple correlated responses on multiple predictors. The model is very flexible, with multiple linear regression and covariance selection models being special cases. However, its practical deployment in genomic data analysis under a Bayesian framework is limited due to both statistical and computational challenges. The statistical challenge is that one needs to infer both the mean vector and the covariance matrix, a problem inherently more complex than separately estimating each. The computational challenge is due to the dimensionality of the parameter space that routinely exceeds the sample size. We propose the use of horseshoe priors on both the mean vector and the inverse covariance matrix. This prior has demonstrated excellent performance when estimating a mean vector or covariance matrix separately. The current work shows these advantages are also present when addressing both simultaneously. A full Bayesian treatment is proposed, with a sampling algorithm that is linear in the number of predictors. MATLAB code implementing the algorithm is freely available from github at this https URL. Extensive performance comparisons are provided with both frequentist and Bayesian alternatives, and both estimation and prediction performances are verified on a genomic data set.
@cite_11: AbstractWe propose a Bayesian procedure for simultaneous variable and covariance selection using continuous spike-and-slab priors in multivariate linear regression models where q possibly correlate...
where @math and @math are the parameters for the spike and slab Laplace distributions, and the binary indicator @math follows a priori a Bernoulli distribution with parameter @math , with a beta hyperprior distribution on @math with parameters @math and @math . Similarly, spike-and-slab lasso priors are put on elements @math in @math as well. An Expectation Conditional Maximization (ECM) algorithm is derived for this model to obtain the posterior mode. The hyper-parameters @math for @math , and the corresponding four hyper-parameters for @math , need to be specified in order to apply the ECM algorithm. In @cite_11 , the Laplace distribution hyper-parameters are chosen by the trajectories of individual parameter estimates given a path of hyper-parameters, and the beta hyper-parameters are set at predefined levels. The method does not provide samples from the full posterior.