Journal of Electrical Engineering and Technology

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지
DOI QR코드

DOI QR Code

Prototype-based Classifier with Feature Selection and Its Design with Particle Swarm Optimization: Analysis and Comparative Studies

  • Received : 2011.01.06
  • Accepted : 2011.12.12
  • Published : 2012.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we introduce a prototype-based classifier with feature selection that dwells upon the usage of a biologically inspired optimization technique of Particle Swarm Optimization (PSO). The design comprises two main phases. In the first phase, PSO selects P % of patterns to be treated as prototypes of c classes. During the second phase, the PSO is instrumental in the formation of a core set of features that constitute a collection of the most meaningful and highly discriminative coordinates of the original feature space. The proposed scheme of feature selection is developed in the wrapper mode with the performance evaluated with the aid of the nearest prototype classifier. The study offers a complete algorithmic framework and demonstrates the effectiveness (quality of solution) and efficiency (computing cost) of the approach when applied to a collection of selected data sets. We also include a comparative study which involves the usage of genetic algorithms (GAs). Numerical experiments show that a suitable selection of prototypes and a substantial reduction of the feature space could be accomplished and the classifier formed in this manner becomes characterized by low classification error. In addition, the advantage of the PSO is quantified in detail by running a number of experiments using Machine Learning datasets.

Keywords

References

  1. F. Fdez-Riverola, E.L. Iglesias, F. Diaz, J.R. Mendez, J.M. Corchado, "SpamHunting: an instance-based reasoning system for spam labeling and filtering," Decision Support Systems, vol. 43, pp. 722-736, 2007. https://doi.org/10.1016/j.dss.2006.11.012
  2. C. Gonzalez, J.F. Lerch, C. Lebiere, "Instance-based learning in dynamic decision making," Cognitive Science, vol. 27, pp. 591-635, 2003. https://doi.org/10.1207/s15516709cog2704_2
  3. C.M. Bishop, Neural networks for Pattern Recognition, Oxford Univ. Press, 1995.
  4. J.-X. Huang, K.-S. Choi, C.-H. Kim, Y.-K. Kim, "Feature-Based Relation Classification Using Quantified Relatedness Information," ETRI Journal, vol. 32, no. 3, pp. 482-485, 2010. https://doi.org/10.4218/etrij.10.0209.0353
  5. X. Wang, J. Yang, X. Teng, W. Xia R. Jensen, "Feature selection based on rough sets and particle swarm optimization," Pattern Recognition, vol. 28, no. 4, pp. 459-471, 2007. https://doi.org/10.1016/j.patrec.2006.09.003
  6. I.-S. Oh, J.-S. Lee, B.-R. Moon, "Hybrid genetic algorithms for feature selection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, pp. 1424- 1437, 2004. https://doi.org/10.1109/TPAMI.2004.105
  7. X. Wang, J. Yang, R. Jensen, X. Liu, "Rough set feature selection and rule induction for prediction of malignancy degree in brain glioma," Computer Methods and Programs in Biomedicine, vol. 83, pp. 147-156, 2006. https://doi.org/10.1016/j.cmpb.2006.06.007
  8. F. Zhu, S. Guan, "Feature selection for modular GAbased classification," Applied Soft Computing, vol. 4, pp. 381-393, 2004. https://doi.org/10.1016/j.asoc.2004.02.001
  9. M.E. Farmer, A.K. Jain, "A wrapper-based approach to image segmentation and classification," IEEE Trans. Image Processing, vol. 14, pp. 2060-2072, 2005. https://doi.org/10.1109/TIP.2005.859374
  10. Y. Liu, Y.F. Zheng, "FS_SFS: A novel feature selection method for support vector machines," Pattern Recognition, vol. 39, pp. 1333-1345, 2006. https://doi.org/10.1016/j.patcog.2005.10.006
  11. J. Kennedy, "The particle swarm: social adaptation of knowledge," Proc. IEEE Int. Conf. Evolutionary Comput, pp. 303-308, 1997.
  12. K.E. Parsopoulos, M.N. Vrahatis, "On the computation of all global minimizers through particle swarm optimization," IEEE Trans. Evolutionary Computation, vol. 8, pp. 211-224, 2004. https://doi.org/10.1109/TEVC.2004.826076
  13. B. Bhanu, Y. Lin, "Genetic algorithm based feature selection for target detection in SAR images," Image and Vision Computing, vol. 21, pp. 591-608, 2003. https://doi.org/10.1016/S0262-8856(03)00057-X
  14. R. Hassan, B. Cohanim, O. de Weck, "A comparison of particle swarm optimization and the genetic algorithm," Proc 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural, Dynamics & Materials Conference, pp. 1-13, 2005.
  15. B. Liu, L. Wang, Y.-H. Jin, F. Tang, D.-X. Huang, "Improved particle swarm optimization combined with chaos," Chaos, Solitons & Fractals, vol. 25, pp. 1261-1271, 2005. https://doi.org/10.1016/j.chaos.2004.11.095
  16. J. Kennedy, W.M. Spears, "Matching algorithms to problems: An experimental test of the particle swarm and some genetic algorithms on multimodal problem generator," Proc IEEE Int Cong Evol Comp, pp. 78- 83, 1998.
  17. A.E. Yilmaz, M. Kuzuoglu, "Calculation of optimized parameters of rectangular microstrip patch antenna using particle swarm optimization," Microwave and Optical Technology Letters, vol. 49, pp.2905- 2907, 2007. https://doi.org/10.1002/mop.22918
  18. E.L. Allwein, R.E. Schapire, "Reducing multiclass to binary: a unifying approach for margin classifiers," The Journal of Machine Learning Research, vol. 1 pp. 113-141, 2001.
  19. S. Dzeroski, B. Zenko, "Stacking with multi-response model trees," Proc. of The Third Int. Workshop on Multiple Classifier Systems, MCS, pp. 201-211, 2002.
  20. T. Li, S. Zhu, M. Ogihara, "Using discriminant analysis for multi-class classification," Third IEEE Int. Conf. on Data Mining ICDM 2003, pp. 589-592, 2003.
  21. A.J. Perez-Jimenez, J.C. Perez-Cortes, "Genetic algorithms for linear feature extraction," Pattern Recognition Letters, vol. 27, pp. 1508-1514, 2006. https://doi.org/10.1016/j.patrec.2006.02.011
  22. X. Zhang, G. Dong, K. Ramamohanarao, "Information- based classification by aggregating emerging patterns, Intelligent Data Engineering and Automated Learning," LNCS, vol. 1983, pp. 48-53, 2000.
  23. C.K. Loo, M.V.C. Rao," Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP," IEEE Trans. Knowledge and Data Engineering, vol. 17, pp. 1589-1593, 2005. https://doi.org/10.1109/TKDE.2005.173
  24. M. Rocha, P. Cortez, J. Neves, "Simultaneous evolution of neural network topologies and weights for classification and regression, Computational Intelligence and Bioinspired Systems," LNCS, vol. 3512, pp. 59-66, 2005.
  25. F. Pernkopf, "Bayesian network classifiers versus selective k-NN classifier," Pattern Recognition, vol. 38, pp. 1-10, 2005. https://doi.org/10.1016/j.patcog.2004.05.012
  26. J.M. Sotoca, J.S. Sanchez, F. Pla, "Attribute relevance in multiclass data sets using the naïve bayes rule," Proc. of the 17th International Conference on Pattern Recognition, vol. 3, pp. 426-429, 2004.
  27. M.A. Tahir, A. Bouridane, F. Kurugollu, "Simultaneous feature selection and feature weighting using hybrid tabu search/k-nearest neighbor classifier," Pattern Recognition Letters, vol. 28, pp. 438-446, 2007. https://doi.org/10.1016/j.patrec.2006.08.016
  28. M. Kudo, J. Sklansky, "Comparison of algorithms that select features for pattern classifiers," Pattern Recognition, vol. 33, pp. 25-41, 2000. https://doi.org/10.1016/S0031-3203(99)00041-2

Cited by

  1. Fuzzy Wavelet Polynomial Neural Networks: Analysis and Design vol.25, pp.5, 2017, https://doi.org/10.1109/TFUZZ.2016.2612267