The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

On the Use of Adaptive Weights for the F∞-Norm Support Vector Machine

  • Received : 2012.07.19
  • Accepted : 2012.09.13
  • Published : 2012.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

When the input features are generated by factors in a classification problem, it is more meaningful to identify important factors, rather than individual features. The $F_{\infty}$-norm support vector machine(SVM) has been developed to perform automatic factor selection in classification. However, the $F_{\infty}$-norm SVM may suffer from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each factor without assessing its relative importance. To overcome such a limitation, we propose the adaptive $F_{\infty}$-norm ($AF_{\infty}$-norm) SVM, which penalizes the empirical hinge loss by the sum of the adaptively weighted factor-wise $L_{\infty}$-norm penalty. The $AF_{\infty}$-norm SVM computes the weights by the 2-norm SVM estimator and can be formulated as a linear programming(LP) problem which is similar to the one of the $F_{\infty}$-norm SVM. The simulation studies show that the proposed $AF_{\infty}$-norm SVM improves upon the $F_{\infty}$-norm SVM in terms of classification accuracy and factor selection performance.

Keywords

References

  1. Bang, S. and Jhun, M. (2012a). Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization, Computational Statistics and Data Analysis, 56, 813-826 . https://doi.org/10.1016/j.csda.2011.01.026
  2. Bang, S. and Jhun, M. (2012b). Adaptive sup-norm regularized simultaneous multiple quantiles regression, Statistics, accepted for publication.
  3. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
  4. Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning, Springer-Verlag, New York.
  5. Hoerl, A. and Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems, Technometrics, 12, 55-67. https://doi.org/10.1080/00401706.1970.10488634
  6. Kaufman, L. and Rousseeuw, P. J. (1990). Finding Groups in Data: An Introduction to Cluster Analysis, Wiley, New York.
  7. Leng, C., Lin, Y. and Wahba, G. (2006). A note on lasso and related procedures in model selection, Statistica Sinica, 16, 1273-1284.
  8. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288.
  9. Vapnik, V. (1995). The Nature of Statistical Learning Theory, Springer-Verlag, New York.
  10. Wang, H. and Leng, C. (2008). A note on adaptive group lasso, Computational Statistics and Data Analysis, 52, 5277-5286. https://doi.org/10.1016/j.csda.2008.05.006
  11. Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society, Series B, 68, 49-67. https://doi.org/10.1111/j.1467-9868.2005.00532.x
  12. Yuan, M. and Lin, Y. (2007). On the nonnegative garrote estimator, Journal of the Royal Statistical Society, Series B, 69, 143-161. https://doi.org/10.1111/j.1467-9868.2007.00581.x
  13. Zhang, H., Liu, Y., Wu, Y. and Zhu, J. (2008). Variable selection for multicategory svm via sup-norm regularization, Electronic Journal of Statistics, 2, 149-167. https://doi.org/10.1214/08-EJS122
  14. Zhao, P., Rocha, G. and Yu, B. (2009). The composite absolute penalties family for grouped and hierarchical variable selection, The Annals of Statistics, 37, 3468-3497. https://doi.org/10.1214/07-AOS584
  15. Zhu, J., Rosset, S., Hastie, T. and Tibshirani, R. (2003). 1-norm support vector machine, Neural Information Proceeding Systems, 16.
  16. Zou, H. (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429. https://doi.org/10.1198/016214506000000735
  17. Zou, H. (2007). An improved 1-norm SVM for simultaneous classification and variable selection, In Proceedings of the 11th International Conference on Arti cial Intelligence and Statistics.
  18. Zou, H. and Yuan, M. (2008). The $F_{\infty}$-norm support vector machine, Statisitca Sinica, 18, 379-398.

Cited by

  1. Hierarchically penalized support vector machine for the classication of imbalanced data with grouped variables vol.29, pp.5, 2016, https://doi.org/10.5351/KJAS.2016.29.5.961