Browse > Article
http://dx.doi.org/10.5351/KJAS.2022.35.3.445

Introduction to variational Bayes for high-dimensional linear and logistic regression models  

Jang, Insong (Department of Statistics, Inha University)
Lee, Kyoungjae (Department of Statistics, Sungkyunkwan University)
Publication Information
The Korean Journal of Applied Statistics / v.35, no.3, 2022 , pp. 445-455 More about this Journal
Abstract
In this paper, we introduce existing Bayesian methods for high-dimensional sparse regression models and compare their performance in various simulation scenarios. Especially, we focus on the variational Bayes approach proposed by Ray and Szabó (2021), which enables scalable and accurate Bayesian inference. Based on simulated data sets from sparse high-dimensional linear regression models, we compare the variational Bayes approach with other Bayesian and frequentist methods. To check the practical performance of the variational Bayes in logistic regression models, a real data analysis is conducted using leukemia data set.
Keywords
variable selection; regression model; spike and slab prior; horseshoe prior;
Citations & Related Records
연도 인용수 순위
  • Reference
1 George E and McCulloch R (1993). Variable selection via Gibbs sampling, Journal of the American Statistical Association, 88, 881-889.   DOI
2 Ishwaran H and Rao J (2005). Spike and slab variable selection: frequentist and Bayesian strategies, The Annals of Statistics, 33, 730-773.   DOI
3 Carvalho C, Polson N, and Scott J (2010). The horseshoe estimator for sparse signals, Biometrika, 97, 465-480.   DOI
4 Makalic E and Schmidt D (2016). A Simple Sampler for the Horseshoe Estimator, IEEE Signal Processing Letters, 23, 179-182.   DOI
5 Ray K, Szabo B, and Clara G (2020). Spike and slab variational Bayes for high dimensional logistic regression, Advances in Neural Information Processing Systems, 33, 14423-14434.
6 Ray K and Szabo (2021). Variational Bayes for High-Dimensional Linear Regression With Sparse Priors, Journal of the American Statistical Association, To appear, 1-12.
7 Golub T, Slonim D, Tamayo P, Huard C, Gaasenbeek M, Mesirov J, Coller H, Loh M, Downing J, Caligiuri M, Bloomfield C, and Lander E (1999). Molecular classification of cancer: class discovery and class prediction by gene expression monitoring, Science, 286, 531-537.   DOI
8 Li L and Yao W (2018). Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection, Journal of Statistical Computation and Simulation, 88, 2827-2851.   DOI
9 McDermott P, Snyder J, and Willison R (2016). Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithm, arXiv: 1605.05429.