Browse > Article
http://dx.doi.org/10.5351/KJAS.2012.25.5.829

On the Use of Adaptive Weights for the F-Norm Support Vector Machine  

Bang, Sung-Wan (Department of Mathematics, Korea Military Academy)
Jhun, Myoung-Shic (Department of Statistics, Korea University)
Publication Information
The Korean Journal of Applied Statistics / v.25, no.5, 2012 , pp. 829-835 More about this Journal
Abstract
When the input features are generated by factors in a classification problem, it is more meaningful to identify important factors, rather than individual features. The $F_{\infty}$-norm support vector machine(SVM) has been developed to perform automatic factor selection in classification. However, the $F_{\infty}$-norm SVM may suffer from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each factor without assessing its relative importance. To overcome such a limitation, we propose the adaptive $F_{\infty}$-norm ($AF_{\infty}$-norm) SVM, which penalizes the empirical hinge loss by the sum of the adaptively weighted factor-wise $L_{\infty}$-norm penalty. The $AF_{\infty}$-norm SVM computes the weights by the 2-norm SVM estimator and can be formulated as a linear programming(LP) problem which is similar to the one of the $F_{\infty}$-norm SVM. The simulation studies show that the proposed $AF_{\infty}$-norm SVM improves upon the $F_{\infty}$-norm SVM in terms of classification accuracy and factor selection performance.
Keywords
Adaptive weight; $F_{\infty}$-norm penalty; factor selection; feature selection; support vector machine;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Bang, S. and Jhun, M. (2012a). Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization, Computational Statistics and Data Analysis, 56, 813-826 .   DOI   ScienceOn
2 Bang, S. and Jhun, M. (2012b). Adaptive sup-norm regularized simultaneous multiple quantiles regression, Statistics, accepted for publication.
3 Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360.   DOI   ScienceOn
4 Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning, Springer-Verlag, New York.
5 Hoerl, A. and Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems, Technometrics, 12, 55-67.   DOI   ScienceOn
6 Kaufman, L. and Rousseeuw, P. J. (1990). Finding Groups in Data: An Introduction to Cluster Analysis, Wiley, New York.
7 Leng, C., Lin, Y. and Wahba, G. (2006). A note on lasso and related procedures in model selection, Statistica Sinica, 16, 1273-1284.
8 Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288.
9 Vapnik, V. (1995). The Nature of Statistical Learning Theory, Springer-Verlag, New York.
10 Wang, H. and Leng, C. (2008). A note on adaptive group lasso, Computational Statistics and Data Analysis, 52, 5277-5286.   DOI   ScienceOn
11 Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society, Series B, 68, 49-67.   DOI   ScienceOn
12 Yuan, M. and Lin, Y. (2007). On the nonnegative garrote estimator, Journal of the Royal Statistical Society, Series B, 69, 143-161.   DOI   ScienceOn
13 Zhang, H., Liu, Y., Wu, Y. and Zhu, J. (2008). Variable selection for multicategory svm via sup-norm regularization, Electronic Journal of Statistics, 2, 149-167.   DOI
14 Zhao, P., Rocha, G. and Yu, B. (2009). The composite absolute penalties family for grouped and hierarchical variable selection, The Annals of Statistics, 37, 3468-3497.   DOI
15 Zhu, J., Rosset, S., Hastie, T. and Tibshirani, R. (2003). 1-norm support vector machine, Neural Information Proceeding Systems, 16.
16 Zou, H. (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429.   DOI
17 Zou, H. (2007). An improved 1-norm SVM for simultaneous classification and variable selection, In Proceedings of the 11th International Conference on Arti cial Intelligence and Statistics.
18 Zou, H. and Yuan, M. (2008). The $F_{\infty}$-norm support vector machine, Statisitca Sinica, 18, 379-398.