Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

An Additive Sparse Penalty for Variable Selection in High-Dimensional Linear Regression Model

  • Lee, Sangin (University of Texas Southwestern Medical Center)
  • Received : 2014.12.30
  • Accepted : 2015.01.29
  • Published : 2015.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

We consider a sparse high-dimensional linear regression model. Penalized methods using LASSO or non-convex penalties have been widely used for variable selection and estimation in high-dimensional regression models. In penalized regression, the selection and prediction performances depend on which penalty function is used. For example, it is known that LASSO has a good prediction performance but tends to select more variables than necessary. In this paper, we propose an additive sparse penalty for variable selection using a combination of LASSO and minimax concave penalties (MCP). The proposed penalty is designed for good properties of both LASSO and MCP.We develop an efficient algorithm to compute the proposed estimator by combining a concave convex procedure and coordinate descent algorithm. Numerical studies show that the proposed method has better selection and prediction performances compared to other penalized methods.

Keywords

References

  1. Chiang, A. P., Beck, J. S., Yen, H. J., Tayeh, M. K., Scheetz, T. E., Swiderski, R. E., Nishimura, D. Y., Braun, T. A., Kim, K. Y., Huang, J., Elbedour, K., Carmi, R., Slusarski, D. C., Casavant, T. L., Stone, E. M. and Sheffield, V. C. (2006). Homozygosity mapping with SNP arrays identifies TRIM32, an E3 ubiquitin ligase, as a Bardet-Biedl syndrome gene (BBS11), Proceedings of the National Academy of Sciences, 103, 6287-6292. https://doi.org/10.1073/pnas.0600158103
  2. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
  3. Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent, Journal of Statistical Software, 33, 1-22.
  4. Kim, Y., Choi, H. and Oh, H.-S. (2008). Smoothly clipped absolute deviation on high dimensions, Journal of the American Statistical Association, 103, 1665-1673. https://doi.org/10.1198/016214508000001066
  5. Scheetz, T. E., Kim, K.-Y. A., Swiderski, R. E., Philp, A. R., Braun, T. A., Knudtson, K. L., Dorrance, A. M., DiBona, G. F., Huang, J., Casavant, T. L., Sheffield, V. C. and Stone, E. M. (2006). Regulations of gene expression in the mammalian eye and its relevance to eye disease, Proceedings of the National Academy of Sciences, 103, 14429-14434. https://doi.org/10.1073/pnas.0602562103
  6. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 58, 267-288.
  7. Wang, H., Li, B. and Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71, 671-683. https://doi.org/10.1111/j.1467-9868.2008.00693.x
  8. Yuille, A. L. and Rangarajan, A. (2003). The concave-convex procedure, Neural Computation, 15, 915-936. https://doi.org/10.1162/08997660360581958
  9. Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty, Annals of Statistics, 58, 894-942.
  10. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67, 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x

Cited by

  1. Model selection algorithm in Gaussian process regression for computer experiments vol.24, pp.4, 2017, https://doi.org/10.5351/CSAM.2017.24.4.383
  2. Penalized rank regression estimator with the smoothly clipped absolute deviation function vol.24, pp.6, 2017, https://doi.org/10.29220/CSAM.2017.24.6.673