Browse > Article
http://dx.doi.org/10.7465/jkdi.2015.26.2.505

A note on standardization in penalized regressions  

Lee, Sangin (Department of Clinical Sciences, University of Texas Southwestern Medical Center)
Publication Information
Journal of the Korean Data and Information Science Society / v.26, no.2, 2015 , pp. 505-516 More about this Journal
Abstract
We consider sparse high-dimensional linear regression models. Penalized regressions have been used as effective methods for variable selection and estimation in high-dimensional models. In penalized regressions, it is common practice to standardize variables before fitting a penalized model and then fit a penalized model with standardized variables. Finally, the estimated coefficients from a penalized model are recovered to the scale on original variables. However, these procedures produce a slightly different solution compared to the corresponding original penalized problem. In this paper, we investigate issues on the standardization of variables in penalized regressions and formulate the definition of the standardized penalized estimator. In addition, we compare the original penalized estimator with the standardized penalized estimator through simulation studies and real data analysis.
Keywords
LASSO; nonconvex penalties; penalized regression; standardization;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Breheny, P. and Huang, J. (2011). Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Annals of Applied Statistics, 5, 232-253.   DOI
2 Chiang, A. P., Beck, J. S., Yen, H. J., Tayeh, M. K., Scheetz, T. E., Swiderski, R. E., Sheffeld, V. C. et al. (2006). Homozygosity mapping with SNP arrays identifies TRIM32, an E3 ubiquitin ligase, as a Bardet-Biedl syndrome gene (BBS11). Proceedings of the National Academy of Sciences, 103, 6287-6292.   DOI   ScienceOn
3 Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32, 407-499.   DOI
4 Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348-1360.   DOI   ScienceOn
5 Fan, J. and Li, R. (2001). Variable selection for Cox's proportional hazards model and frailty model. Annals of Statistics, 30, 74-99.
6 Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9, 432-441.   DOI   ScienceOn
7 Friedman, J., Hastie, T. and Tibshirani, R. (2008). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33, 1-22.
8 Kim, Y., Choi, H. and Oh, H. S. (2008). Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103, 1665-1673.   DOI   ScienceOn
9 Kim, Y. and Kwon, S. (2012). Global optimality of nonconvex penalized estimators. Biometrika, 99, 315-325.   DOI
10 Kwon, S., Han, S. and Lee, S. (2013). A small review and further studies on the lasso. Journal of the Korean Data & Information Science Society, 24, 1077-1088.   DOI   ScienceOn
11 Park, C. (2013). Simple principal component analysis using lasso. Journal of the Korean Data & Information Science Society, 24, 533-541.   DOI   ScienceOn
12 Scheetz, T. E., Kim, K. Y. A., Swiderski, R. E., Philp, A. R., Braun, T. A., Knudtson, K. L., Dibona, G. F., Stone, E. M. et al. (2006). Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proceedings of the National Academy of Sciences, 103, 14429-14434.   DOI   ScienceOn
13 Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society B, 58, 267-288.
14 Tseng, P. (2001). Convergence of a block coordinate descent method for nondi erentiable minimization. Journal of Optimization Theory and Applications, 109, 475-494.   DOI   ScienceOn
15 Van de Geer, S. A. (2008). High-dimensional generalized linear models and the lasso. Annals of Statistics, 36, 614-645.   DOI
16 Yuille, A. L. and Rangarajan, A. (2003). The concave-convex procedure. Neural Computation, 15, 915-936.   DOI   ScienceOn
17 Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38, 894-942.   DOI