Browse > Article
http://dx.doi.org/10.3745/KIPSTB.2007.14-B.4.303

Ensemble Learning of Region Based Classifiers  

Choi, Sung-Ha (서강대학교 대학원 컴퓨터학과)
Lee, Byung-Woo (서강대학교 대학원 컴퓨터학과)
Yang, Ji-Hoon (서강대학교 컴퓨터학과)
Publication Information
The KIPS Transactions:PartB / v.14B, no.4, 2007 , pp. 303-310 More about this Journal
Abstract
In machine learning, the ensemble classifier that is a set of classifiers have been introduced for higher accuracy than individual classifiers. We propose a new ensemble learning method that employs a set of region based classifiers. To show the performance of the proposed method. we compared its performance with that of bagging and boosting, which ard existing ensemble methods. Since the distribution of data can be different in different regions in the feature space, we split the data and generate classifiers based on each region and apply a weighted voting among the classifiers. We used 11 data sets from the UCI Machine Learning Repository to compare the performance of our new ensemble method with that of individual classifiers as well as existing ensemble methods such as bagging and boosting. As a result, we found that our method produced improved performance, particularly when the base learner is Naive Bayes or SVM.
Keywords
Machine Learning; Ensemble; Bagging; Boosting;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Dietterich, T., 'Ensemble method in Machine learning', In J. Kittler and F. Roli (Ed.) First International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science, pp. 1-15, 2000   DOI   ScienceOn
2 Bauer, E. & Kohavi, R., 'An Empirical Comparison of Voting Classification Algorithm: Bagging, Boosting, and Variants', Machine Learning, 36(1-2), pp. 105-142, 1999   DOI
3 Blake, C. & Merz, C., UCI Repository of Machine Learning Database, http//www.ics.uci.edu /~mlearn/MLRepository.html, 1998
4 Breiman, L., 'Bias, Variance, and Arcing Classifiers', Technical Report TR, 460, UC Berkeley, 1996
5 Quinlan, J., 'Bagging, Boosting, and C4.5.', In Proc. of the Thirteenth National Conference on Artificial Intelligence, pp. 725-730, 1996
6 Friedman, J., Hastie, T. & Tibshirani, R., 'Additive Logistic Regression: a Statistical View of Boosting', Annals of Statistics, 28(2), pp. 337-374, 2000   DOI
7 Dietterich, T., 'Ensemble Learning', In The Handbook of Brain Theory and Neural Networks, Second edition, The MIT Press, pp. 405-408, 2002
8 Freund, Y. & Schapire, R., 'Experiments with a new boosting algorithm', In Proc. of the Thirteenth International Conference on Machine Learning, pp. 148-156, 1996
9 Freund, Y. & Schapire, R., 'A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting', Journal of Computer and System Science, 55, pp. 119-139, 1997   DOI   ScienceOn
10 L.I. Kuncheva and C.J. Whitaker. 'Measures of diversity in classifier ensembles', Machine Learning, 51, pp. 181-207, 2003   DOI
11 Opitz, D. & Maclin, R., 'Popular Ensemble Methods: An Empirical Study', Journal of Artificial Intelligence Research, 11, pp. 169-198, 1999   DOI
12 Breiman, L., 'Bagging Predictors', Machine Learning, 24(2), pp. 123-140, 1996   DOI
13 Platt, J. Fast Training of Support Vector Machines using Sequential Minimal Optimization, chapter 12, pp. 185-208, The MIT Press, 1999
14 Quinlan, J., 'Induction of Decision Tree', Machine Learning, 1(1), pp. 81- 106, 1986   DOI
15 Quinlan, J., C4.5: Programs for Machine Learning, Morgan Kaufmann, 1993
16 Dietterich, T., 'An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization', Machine Learning , 40(2), pp. 139-157, 2000   DOI
17 Hansen, L. & Salamon, P., 'Neural Network Ensembles', IEEE Transaction on Pattern Analysis and Machine Intelligence, 12, pp. 993-1001, 1990   DOI   ScienceOn
18 Witten, I. & Frank, E., Data Mining: Practical Machine Learning Tools and Techniques with Java Implementation, Second edition, Morgan Kaufmann, 2005