Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Principal Component Regression by Principal Component Selection

  • Lee, Hosung (Department of Statistics, Hankuk University of Foreign Studies) ;
  • Park, Yun Mi (Department of Statistics, Hankuk University of Foreign Studies) ;
  • Lee, Seokho (Department of Statistics, Hankuk University of Foreign Studies)
  • Received : 2015.01.23
  • Accepted : 2015.02.27
  • Published : 2015.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a selection procedure of principal components in principal component regression. Our method selects principal components using variable selection procedures instead of a small subset of major principal components in principal component regression. Our procedure consists of two steps to improve estimation and prediction. First, we reduce the number of principal components using the conventional principal component regression to yield the set of candidate principal components and then select principal components among the candidate set using sparse regression techniques. The performance of our proposals is demonstrated numerically and compared with the typical dimension reduction approaches (including principal component regression and partial least square regression) using synthetic and real datasets.

Keywords

References

  1. Bishop, C. M. (2006). Pattern Recognition and Machine Learning, Springer.
  2. Byrd, A. (2005). Penalized principal component regression, Master thesis, University of Georgia.
  3. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
  4. Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Element of Statistical Learning: Data Mining, Inference, and Prediction, The 2nd Edition, Springer.
  5. Kim, K. and Lee, S. (2014). Logistic regression classification by principal component selection, Communications for Statistical Applications and Methods, 21, 61-68. https://doi.org/10.5351/CSAM.2014.21.1.061
  6. Lee, H. (2015). On the estimation for sparse principal component regression approach under multiple regression problem, Master thesis, Hankuk University of Foreign Studies.
  7. Lichman, M. (2013). UCI Machine Learning Repository (http://archive.ics.uci.edu/ml), Irvine, University of California, School of Information and Computer Science, California.
  8. Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective, The MIT Press.
  9. Park, Y. M. (2015). Logistic Principal Component Regression based on the sparse method, Master thesis, Hankuk University of Foreign Studies.
  10. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288.
  11. Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty, The Annals of Statistics, 38, 894-942. https://doi.org/10.1214/09-AOS729