Browse > Article
http://dx.doi.org/10.5370/JEET.2012.7.2.245

Prototype-based Classifier with Feature Selection and Its Design with Particle Swarm Optimization: Analysis and Comparative Studies  

Park, Byoung-Jun (Electronics and Telecommunications Research Institute)
Oh, Sung-Kwun (Dep. of Electrical Engineering, The University of Suwon)
Publication Information
Journal of Electrical Engineering and Technology / v.7, no.2, 2012 , pp. 245-254 More about this Journal
Abstract
In this study, we introduce a prototype-based classifier with feature selection that dwells upon the usage of a biologically inspired optimization technique of Particle Swarm Optimization (PSO). The design comprises two main phases. In the first phase, PSO selects P % of patterns to be treated as prototypes of c classes. During the second phase, the PSO is instrumental in the formation of a core set of features that constitute a collection of the most meaningful and highly discriminative coordinates of the original feature space. The proposed scheme of feature selection is developed in the wrapper mode with the performance evaluated with the aid of the nearest prototype classifier. The study offers a complete algorithmic framework and demonstrates the effectiveness (quality of solution) and efficiency (computing cost) of the approach when applied to a collection of selected data sets. We also include a comparative study which involves the usage of genetic algorithms (GAs). Numerical experiments show that a suitable selection of prototypes and a substantial reduction of the feature space could be accomplished and the classifier formed in this manner becomes characterized by low classification error. In addition, the advantage of the PSO is quantified in detail by running a number of experiments using Machine Learning datasets.
Keywords
Prototypes; Feature selection; Particle Swarm Optimization (PSO); Wrapper mode of feature selection; Classification; Computational Intelligence (CI);
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
Times Cited By Web Of Science : 0  (Related Records In Web of Science)
연도 인용수 순위
1 F. Pernkopf, "Bayesian network classifiers versus selective k-NN classifier," Pattern Recognition, vol. 38, pp. 1-10, 2005.   DOI   ScienceOn
2 J.M. Sotoca, J.S. Sanchez, F. Pla, "Attribute relevance in multiclass data sets using the naïve bayes rule," Proc. of the 17th International Conference on Pattern Recognition, vol. 3, pp. 426-429, 2004.
3 M.A. Tahir, A. Bouridane, F. Kurugollu, "Simultaneous feature selection and feature weighting using hybrid tabu search/k-nearest neighbor classifier," Pattern Recognition Letters, vol. 28, pp. 438-446, 2007.   DOI   ScienceOn
4 M. Kudo, J. Sklansky, "Comparison of algorithms that select features for pattern classifiers," Pattern Recognition, vol. 33, pp. 25-41, 2000.   DOI   ScienceOn
5 B. Liu, L. Wang, Y.-H. Jin, F. Tang, D.-X. Huang, "Improved particle swarm optimization combined with chaos," Chaos, Solitons & Fractals, vol. 25, pp. 1261-1271, 2005.   DOI   ScienceOn
6 J. Kennedy, W.M. Spears, "Matching algorithms to problems: An experimental test of the particle swarm and some genetic algorithms on multimodal problem generator," Proc IEEE Int Cong Evol Comp, pp. 78- 83, 1998.
7 A.E. Yilmaz, M. Kuzuoglu, "Calculation of optimized parameters of rectangular microstrip patch antenna using particle swarm optimization," Microwave and Optical Technology Letters, vol. 49, pp.2905- 2907, 2007.   DOI   ScienceOn
8 E.L. Allwein, R.E. Schapire, "Reducing multiclass to binary: a unifying approach for margin classifiers," The Journal of Machine Learning Research, vol. 1 pp. 113-141, 2001.
9 S. Dzeroski, B. Zenko, "Stacking with multi-response model trees," Proc. of The Third Int. Workshop on Multiple Classifier Systems, MCS, pp. 201-211, 2002.
10 T. Li, S. Zhu, M. Ogihara, "Using discriminant analysis for multi-class classification," Third IEEE Int. Conf. on Data Mining ICDM 2003, pp. 589-592, 2003.
11 M. Rocha, P. Cortez, J. Neves, "Simultaneous evolution of neural network topologies and weights for classification and regression, Computational Intelligence and Bioinspired Systems," LNCS, vol. 3512, pp. 59-66, 2005.
12 A.J. Perez-Jimenez, J.C. Perez-Cortes, "Genetic algorithms for linear feature extraction," Pattern Recognition Letters, vol. 27, pp. 1508-1514, 2006.   DOI   ScienceOn
13 X. Zhang, G. Dong, K. Ramamohanarao, "Information- based classification by aggregating emerging patterns, Intelligent Data Engineering and Automated Learning," LNCS, vol. 1983, pp. 48-53, 2000.
14 C.K. Loo, M.V.C. Rao," Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP," IEEE Trans. Knowledge and Data Engineering, vol. 17, pp. 1589-1593, 2005.   DOI   ScienceOn
15 J.-X. Huang, K.-S. Choi, C.-H. Kim, Y.-K. Kim, "Feature-Based Relation Classification Using Quantified Relatedness Information," ETRI Journal, vol. 32, no. 3, pp. 482-485, 2010.   DOI   ScienceOn
16 X. Wang, J. Yang, X. Teng, W. Xia R. Jensen, "Feature selection based on rough sets and particle swarm optimization," Pattern Recognition, vol. 28, no. 4, pp. 459-471, 2007.   DOI   ScienceOn
17 I.-S. Oh, J.-S. Lee, B.-R. Moon, "Hybrid genetic algorithms for feature selection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, pp. 1424- 1437, 2004.   DOI   ScienceOn
18 X. Wang, J. Yang, R. Jensen, X. Liu, "Rough set feature selection and rule induction for prediction of malignancy degree in brain glioma," Computer Methods and Programs in Biomedicine, vol. 83, pp. 147-156, 2006.   DOI   ScienceOn
19 F. Zhu, S. Guan, "Feature selection for modular GAbased classification," Applied Soft Computing, vol. 4, pp. 381-393, 2004.   DOI   ScienceOn
20 M.E. Farmer, A.K. Jain, "A wrapper-based approach to image segmentation and classification," IEEE Trans. Image Processing, vol. 14, pp. 2060-2072, 2005.   DOI   ScienceOn
21 Y. Liu, Y.F. Zheng, "FS_SFS: A novel feature selection method for support vector machines," Pattern Recognition, vol. 39, pp. 1333-1345, 2006.   DOI   ScienceOn
22 J. Kennedy, "The particle swarm: social adaptation of knowledge," Proc. IEEE Int. Conf. Evolutionary Comput, pp. 303-308, 1997.
23 K.E. Parsopoulos, M.N. Vrahatis, "On the computation of all global minimizers through particle swarm optimization," IEEE Trans. Evolutionary Computation, vol. 8, pp. 211-224, 2004.   DOI   ScienceOn
24 B. Bhanu, Y. Lin, "Genetic algorithm based feature selection for target detection in SAR images," Image and Vision Computing, vol. 21, pp. 591-608, 2003.   DOI   ScienceOn
25 R. Hassan, B. Cohanim, O. de Weck, "A comparison of particle swarm optimization and the genetic algorithm," Proc 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural, Dynamics & Materials Conference, pp. 1-13, 2005.
26 C.M. Bishop, Neural networks for Pattern Recognition, Oxford Univ. Press, 1995.
27 F. Fdez-Riverola, E.L. Iglesias, F. Diaz, J.R. Mendez, J.M. Corchado, "SpamHunting: an instance-based reasoning system for spam labeling and filtering," Decision Support Systems, vol. 43, pp. 722-736, 2007.   DOI   ScienceOn
28 C. Gonzalez, J.F. Lerch, C. Lebiere, "Instance-based learning in dynamic decision making," Cognitive Science, vol. 27, pp. 591-635, 2003.   DOI   ScienceOn