Browse > Article

Coarse-to-fine Classifier Ensemble Selection using Clustering and Genetic Algorithms  

Kim, Young-Won (한국전자통신연구원 우정기술연구센터 자동구분처리연구팀)
Oh, Il-Seok (전북대학교 전자정보공학부)
Publication Information
Journal of KIISE:Software and Applications / v.34, no.9, 2007 , pp. 857-868 More about this Journal
Abstract
The good classifier ensemble should have a high complementarity among classifiers in order to produce a high recognition rate and its size is small in order to be efficient. This paper proposes a classifier ensemble selection algorithm with coarse-to-fine stages. for the algorithm to be successful, the original classifier pool should be sufficiently diverse. This paper produces a large classifier pool by combining several different classification algorithms and lots of feature subsets. The aim of the coarse selection is to reduce the size of classifier pool with little sacrifice of recognition performance. The fine selection finds near-optimal ensemble using genetic algorithms. A hybrid genetic algorithm with improved searching capability is also proposed. The experimentation uses the worldwide handwritten numeral databases. The results showed that the proposed algorithm is superior to the conventional ones.
Keywords
Classifier Ensemble Selection; Genetic Algorithm; Classifier Diversity; Classifier Clustering; Handwritten Numeral Classification;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 G. Fumera and F. Roli, 'A theoretical and experimental analysis of linear combiners for multiple classifier systems,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.27, no.6, pp.942-956, 2005   DOI
2 J. Kittler and F.M. Alkoot, 'Sum versus vote fusion in multiple classifier systems,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.25, pp.110-115, 2003   DOI   ScienceOn
3 Il-Seok Oh and Ching Y. Suen, 'Distance features for neural network-based recognition of handwritten characters,' International Journal on Document Analysis and Recognition, vol.1, pp.73-88, 1998   DOI   ScienceOn
4 Il-Seok Oh, Jin-Seon Lee, and Byung-Ro Moon, 'Hybrid genetic algorithms for feature selection,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.26, no.11, pp.1424-1437, 2004   DOI   ScienceOn
5 J.J. Rodriguez, L.I. Kuncheva, and C.J. Alonso, 'Rotation forest: a new classifier ensemble method,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.28, no.10, pp.1619-1630, October 2006   DOI
6 A.J.C. Sharkey, N.E. Sharkey, U. Gerecke, and G.O. Chandroth, 'The test and select approach to ensemble combination,' in Multiple Classifier Systems (Ed. by J. Kittler and F. Roli), Springer, 2000
7 S.Y. Sohn and H.W. Shin, 'Experimental study for the comparison of classifier combination methods,' Pattern Recognition, Vol.40, pp.33-40, 2007   DOI   ScienceOn
8 S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd ed., Academic Press, 2006
9 N. Ueda, 'Optimal linear combination of neural networks for improving classification performance,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.22, no.2, pp.207-215, 2000   DOI   ScienceOn
10 N.M. Wanas, R.A. Dara, M.S. Kamel, 'Adaptive fusion and co-operative training for classifier ensembles,' Pattern Recognition, Vol.39, pp.1781-1794, 2006   DOI   ScienceOn
11 이진선, 김영원, 오일석, '대용량 분류에서 SVM과 신경망의 성능 비교,' 정보처리 학회 논문지, vol.12-B, no.1, pp.25-30, 2005   과학기술학회마을   DOI
12 G. Giacinto and F Roli, 'An approach to the automatic design of multiple classifier systems,' Pattern Recognition Letters, vol.22, pp.25-33, 2001   DOI   ScienceOn
13 R.E. Banfield, L.O. Hall, K.W. Bowyer, and W.P. Kegelmeyer, 'A comparison of decision tree ensemble creation method,' IEEE Tr. Pattern Analysis and Machine Intelligence, vol.29, no.1, pp.173-180, January 2007   DOI
14 Tin Kam Ho, 'Multiple classifier combination: lessons and next steps,' in Hybrid Methods in Pattern Recognition, (Ed. by H. Bubke & A. Kandel), pp.171-198, World Scientific, 2002
15 Zhi-Hua Zhou, Jianxin Wu, and Wei Tang, 'Ensembling neural networks: many could be better than all,' Artificial Intelligence, vol.137, pp.239-263, 2002   DOI   ScienceOn
16 P.M. Granitto, P.F. Verdes, and H.A. Ceccatto, 'Neural network ensembles: evaluation of aggregation algorithms,' Artificial Intelligence, vol.163, no.2, pp.139-162, 2005   DOI   ScienceOn
17 H. Hao, C.-L. Liu, and H. Sako, 'Comparison of genetic algorithm and sequential search methods for classifier subset selection,' Proceedings of ICDAR, 2003
18 C.-L. Liu, K. Nakashima, H. Sako, and H. Fujisawa, 'Handwritten digit recognition: benchmarking of state-of-the-art techniques,' Pattern Recognition, Vol.36, pp.2271-2285, 2004   DOI   ScienceOn
19 http://svmlight.joachims.org, 2007
20 문병로, 유전알고리즘, 두양사, 2003
21 D. Partridge and W. B. Yates, 'Engineering multiversion neural-net systems,' Neural Computation, vol.8, pp.869-893, 1996   DOI   ScienceOn
22 N. Garcia-Pedrajas, C. Hervas-Martinez, and D. Ortiz-Boyer, 'Cooperative coevolution of artificial neural network ensembles for pattern classification,' IEEE Tr. Evolutionary Computation, Vol.9, No.3, pp.271-302, June 2005   DOI   ScienceOn