Browse > Article
http://dx.doi.org/10.5351/KJAS.2021.34.3.491

A comparison study of Bayesian high-dimensional linear regression models  

Shin, Ju-Won (Department of Statistics, Inha University)
Lee, Kyoungjae (Department of Statistics, Inha University)
Publication Information
The Korean Journal of Applied Statistics / v.34, no.3, 2021 , pp. 491-505 More about this Journal
Abstract
We consider linear regression models in high-dimensional settings (p ≫ n) and compare various classes of priors. The spike and slab prior is one of the most widely used priors for Bayesian regression models, but its model space is vast, resulting in a bad performance in finite samples. As an alternative, various continuous shrinkage priors, including the horseshoe prior and its variants, have been proposed. Although each of the above priors has been investigated separately, exhaustive comparative studies of their performance have been conducted very rarely. In this study, we compare the spike and slab prior, the horseshoe prior and its variants in various simulation settings. The performance of each method is demonstrated in terms of the regression coefficient estimation and variable selection. Finally, some remarks and suggestions are given based on comprehensive simulation studies.
Keywords
Bayesian regression; spike and slab prior; continuous shrinkage prior;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Van ES, Oberski DL, and Mulder J (2019). Shrinkage priors for Bayesian penalized regression, Journal of Mathematical Psychology, 89, 31-50.   DOI
2 George EI and McCulloch RE (1993). Variable selection via gibbs sampling, Journal of the American Statistical Association, 88, 881-889.   DOI
3 Ishwaran H and Rao JS (2005). Spike and slab variable selection: frequentist and Bayesian strategies, The Annals of Statistics, 33, 730-773.   DOI
4 Lee SY, Pati D, and Mallick BK (2020). Continuous Shrinkage Prior Revisited: A Collapsing Behavior and Remedy, arXiv preprint arXiv:2007.02192.
5 Makalic E and Schmidt DF (2016). High-Dimensional Bayesian Regularised Regression with the Bayesreg Package, arXiv preprint arXiv:1611.06649.
6 Martin R, Mess R, and Walker SG (2017). Empirical bayes posterior concentration in sparse high-dimensional linear models, Bernoulli, 23, 1822-1847.
7 Piironen J and Vehtari A (2017) . Sparsity information and regularization in the horseshoe and other shrinkage priors, Electronic Journal of Statistics, 11, 5018-5051.   DOI
8 Tibshirani R (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society: Series B (Methodological), 58, 267-288.   DOI
9 Barbieri MM, Berger JO (2004). Optimal predictive model selection, The Annals of Statistics, 32, 870-897.   DOI
10 Bhadra A, Datta J, Polson NG, and Willard B (2017). The horseshoe+ estimator of ultra-sparse signals, Bayesian Analysis, 12, 1105-1131.   DOI
11 Carvalho CM, Polson NG, and Scott JG (2010). The horseshoe estimator for sparse signals, Biometrika, 97, 465-480.   DOI
12 Hoerl AE and Kennard RW (1970). Ridge regression: applications to nonorthogonal problem, Technometrics, 12, 69-82.   DOI
13 Makalic E and Schmidt DF (2015). A simple sampler for the horseshoe estimator, IEEE Signal Processing Letters, 23, 179-182.   DOI
14 Piironen J and Vehtari A (2017). On the hyperprior choice for the global shrinkage parameter in the horseshoe prior. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 905-913.