Browse > Article
http://dx.doi.org/10.3745/KIPSTB.2008.15-B.4.331

Performance Improvement of Feature Selection Methods based on Bio-Inspired Algorithms  

Yun, Chul-Min ((주)다이퀘스트)
Yang, Ji-Hoon (서강대학교 컴퓨터학과)
Publication Information
The KIPS Transactions:PartB / v.15B, no.4, 2008 , pp. 331-340 More about this Journal
Abstract
Feature Selection is one of methods to improve the classification accuracy of data in the field of machine learning. Many feature selection algorithms have been proposed and discussed for years. However, the problem of finding the optimal feature subset from full data still remains to be a difficult problem. Bio-inspired algorithms are well-known evolutionary algorithms based on the principles of behavior of organisms, and very useful methods to find the optimal solution in optimization problems. Bio-inspired algorithms are also used in the field of feature selection problems. So in this paper we proposed new improved bio-inspired algorithms for feature selection. We used well-known bio-inspired algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO), to find the optimal subset of features that shows the best performance in classification accuracy. In addition, we modified the bio-inspired algorithms considering the prior importance (prior relevance) of each feature. We chose the mRMR method, which can measure the goodness of single feature, to set the prior importance of each feature. We modified the evolution operators of GA and PSO by using the prior importance of each feature. We verified the performance of the proposed methods by experiment with datasets. Feature selection methods using GA and PSO produced better performances in terms of the classification accuracy. The modified method with the prior importance demonstrated improved performances in terms of the evolution speed and the classification accuracy.
Keywords
Machine Learning; Feture Selection; Bio-Inspired Algorithm; Evolutionary Algorithms;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Peng, H., Long, F. and Ding, C., “Feature Selection based on Mutual Information : Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.8, pp.1226-1238, 2005   DOI   ScienceOn
2 Zhao, J., Wang, G., Wu, Z., Tang, H. and Li, H., “The Study on Technologies for Feature Selection,” Proceedings of the First International Conference on Machine Learning and Cybernetics, pp.689-693, 2002
3 Yang, J. and Honavar, V., “Feature Subset Selection Using a Genetic Algorithm,” IEEE Intelligent Systems, Vol.13, pp.44-49, 1998   DOI   ScienceOn
4 Muni, D.P., Pal, N.R. and Das, J. “Genetic programming for simultaneous feature selection and classifier design,” IEEE Transactions on Systems, Man, and Cybernetics, Vol.36, pp. 106-117, 2006   DOI   ScienceOn
5 Bello, R., Gomez, Y., Nowe, A. and Garcia, Maria M. “Two-Step Particle Swarm Optimization to Solve the Feature Selection Problem,” Proceedings of Seventh International Conference on Intelligent Systems Design and Applications, pp.691-696, 2007
6 Geetha, K., Thanushkodi, K. and Kumar, A.K. “New Particle Swarm Optimization for Feature Selection and Classification of Microcalcifications in Mammograms,” Proceedings of International Conference on Signal Processing, Communications and Networking, pp.458-463, 2008
7 Goldberg, D., Genetic Algorithms in Search, Optimization, and Machine Learning, ADDISON-WESLEY, 1989
8 Kennedy, J. and Eberhart, R., Swarm Intelligence, MORGAN KAUFMANN, 2001
9 Kennedy, J. and Eberhart, R., “Particle Swarm Optimization,” Proceedings of the Conference on Neural Networks, pp.1942-1948, 1995
10 Engelbrecht, A., Fundamentals of Computational Swarm Intelligence. WILEY, 2005
11 Quinlan, J., C4.5: Programs for Machine Learning, MORGAN KAUFMANN, 1993
12 Blake, C. and Merz, C., UCI Repository of Machine Learning Database, http://www.ics.uci.edu/~mlearn/MLRepository.html, 1998
13 Mitchell, T., Machine Learning, MCGRAW-HILL, 1997
14 Platt, C., “Fast Training of Support Vector Machines using Sequential Minimal Optimization,” in Advances in Kernel Methods: Support Vector Learning, pp.185-208, 1999
15 WEKA3.5, http://www.cs.waikato.ac.nz/~ml/
16 Blum, A. and Langley, P., “Selection of Relevant Features and Examples in Machine Learning,” Artificial Intelligence, Vol.97, No.1-2, pp.245-271, 1997   DOI   ScienceOn
17 Liu, H. and Yu, L., “Toward Integrating Feature Selection Algorithms for Classification and Clustering,” IEEE Transactions on Knowledge and Data Engineering, Vol.17, No.4, pp.491-502, 2005   DOI   ScienceOn
18 Jain, A. and Zongker, D., “Feature Selection : Evaluation, Application, and Small Sample Performance,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.2, pp.153-158, 1997   DOI   ScienceOn
19 Mitchell, M., An Introduction to Genetic Algorithms, MIT PRESS, 1996
20 Bautista, M. and Vila, M., “A Survey of Genetic Feature Selection in Mining Issues,” Proceedings of the Congress on Evolutionary Computation, Vol.2, pp.1314-1321, 1999   DOI
21 Galbally, J., Fierrez, J., Freire, M.R. and Ortega-Garcia, J., “Feature Selection Based on Genetic Algorithms for On-Line Signature Verification,” Proceedings of IEEE Workshop on Automatic Identification Advanced Technologies, pp.198-203, 2007   DOI
22 Liu, Y., Qin, Z., Xu, Z. and He, X., “Feature Selection with Particle Swarms,” Proceedings of International Symposium on Computational and Information Science, pp.425-430, 2004
23 Firpi, A. and Goodman, E., “Swarmed Feature Selection,” Proceedings of the 33rd Applied Imagery Pattern Recognition Workshop, pp.112-118, 2004
24 Yan, Z. and Yuan, C., “Ant Colony Optimization for Feature Selection in Face Recognition,” Lecture Note in Computer Science, Vol.3072, SPRINGER, 2004