Journal of Digital Convergence (디지털융복합연구)

The Society of Digital Policy and Management (한국디지털정책학회)
권호 표지
DOI QR코드

DOI QR Code

Study for Feature Selection Based on Multi-Agent Reinforcement Learning

다중 에이전트 강화학습 기반 특징 선택에 대한 연구

  • Kim, Miin-Woo (Division of IT Convergence Engineering, Gachon University) ;
  • Bae, Jin-Hee (Division of IT Convergence Engineering, Gachon University) ;
  • Wang, Bo-Hyun (Division of Computer Engineering, Gachone University) ;
  • Lim, Joon-Shik (Division of Computer Engineering, Gachone University)
  • 김민우 (가천대학교 IT융합공학과) ;
  • 배진희 (가천대학교 IT융합공학과) ;
  • 왕보현 (가천대학교 컴퓨터공학과) ;
  • 임준식 (가천대학교 컴퓨터공학과)
  • Received : 2021.10.31
  • Accepted : 2021.12.20
  • Published : 2021.12.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method for finding feature subsets that are effective for classification in an input dataset by using a multi-agent reinforcement learning method. In the field of machine learning, it is crucial to find features suitable for classification. A dataset may have numerous features; while some features may be effective for classification or prediction, others may have little or rather negative effects on results. In machine learning problems, feature selection for increasing classification or prediction accuracy is a critical problem. To solve this problem, we proposed a feature selection method based on reinforced learning. Each feature has one agent, which determines whether the feature is selected. After obtaining corresponding rewards for each feature that is selected, but not by the agents, the Q-value of each agent is updated by comparing the rewards. The reward comparison of the two subsets helps agents determine whether their actions were right. These processes are performed as many times as the number of episodes, and finally, features are selected. As a result of applying this method to the Wisconsin Breast Cancer, Spambase, Musk, and Colon Cancer datasets, accuracy improvements of 0.0385, 0.0904, 0.1252 and 0.2055 were shown, respectively, and finally, classification accuracies of 0.9789, 0.9311, 0.9691 and 0.9474 were achieved, respectively. It was proved that our proposed method could properly select features that were effective for classification and increase classification accuracy.

본 논문은 다중 에이전트 강화학습 방식을 사용하여 입력 데이터로부터 분류에 효과적인 특징 집합을 찾아내는 방식을 제안한다. 기계 학습 분야에 있어서 분류에 적합한 특징들을 찾아내는 것은 매우 중요하다. 데이터에는 수많은 특징들이 존재할 수 있으며, 여러 특징들 중 일부는 분류나 예측에 효과적일 수 있지만 다른 특징들은 잡음 역할을 함으로써 올바른 결과를 생성하는 데에 오히려 악영향을 줄 수 있다. 기계 학습 문제에서 분류나 예측 정확도를 높이기 위한 특징 선택은 매우 중요한 문제 중 하나이다. 이러한 문제를 해결하기 위해 강화학습을 통한 특징 선택 방법을 제시한다. 각각의 특징들은 하나의 에이전트를 가지게 되며, 이 에이전트들은 특징을 선택할 것인지 말 것인지에 대한 여부를 결정한다. 에이전트들에 의해 선택된 특징들과 선택되지 않은 특징들에 대해서 각각 보상을 구한 뒤, 보상에 대한 비교를 통해 에이전트의 Q-value 값을 업데이트 한다. 두 하위 집합에 대한 보상 비교는 에이전트로 하여금 자신의 행동이 옳은지에 대한 판단을 내릴 수 있도록 도와준다. 이러한 과정들을 에피소드 수만큼 반복한 뒤, 최종적으로 특징들을 선별한다. 이 방법을 통해 Wisconsin Breast Cancer, Spambase, Musk, Colon Cancer 데이터 세트에 적용한 결과, 각각 0.0385, 0.0904, 0.1252, 0.2055의 정확도 향상을 보여주었으며, 최종적으로 0.9789, 0.9311, 0.9691, 0.9474의 분류 정확도를 보여주었다. 이는 우리가 제안한 방법이 분류에 효과적인 특징들을 잘 선별하고 분류에 대한 정확도를 높일 수 있음을 보여준다.

Keywords

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(2020R1I1A1A01066599), This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2021-2017-0-01630) supervised by the IITP(Institute for Information & communications Technology Promotion)

References

  1. Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of machine learning research, 3(Mar), 1157-1182.
  2. Cai, J., Luo, J., Wang, S., & Yang, S. (2018). Feature selection in machine learning: A new perspective. Neurocomputing, 300, 70-79. https://doi.org/10.1016/j.neucom.2017.11.077
  3. Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16-28. https://doi.org/10.1016/j.compeleceng.2013.11.024
  4. Dash, M., & Liu, H. (1997). Feature selection for classification. Intelligent data analysis, 1(1-4), 131-156. https://doi.org/10.3233/IDA-1997-1302
  5. Kira, K., & Rendell, L. A. (1992, July). The feature selection problem: Traditional methods and a new algorithm. In Aaai (Vol. 2, No. 1992a, pp. 129-134).
  6. Bousquet, O., Boucheron, S., & Lugosi, G. (2003, February). Introduction to statistical learning theory. In Summer school on machine learning (pp. 169-207). Springer, Berlin, Heidelberg.
  7. Bermingham, M. L., Pong-Wong, R., Spiliopoulou, A., Hayward, C., Rudan, I., Campbell, H., ... & Haley, C. S. (2015). Application of high-dimensional feature selection: evaluation for genomic prediction in man. Scientific reports, 5(1), 1-12. https://doi.org/10.9734/JSRR/2015/14076
  8. Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on pattern analysis and machine intelligence, 27(8), 1226-1238. https://doi.org/10.1109/TPAMI.2005.159
  9. Raileanu, L. E., & Stoffel, K. (2004). Theoretical comparison between the gini index and information gain criteria. Annals of Mathematics and Artificial Intelligence, 41(1), 77-93. https://doi.org/10.1023/B:AMAI.0000018580.96245.c6
  10. Kira, K., & Rendell, L. A. (1992). A practical approach to feature selection. In Machine learning proceedings 1992 (pp. 249-256). Morgan Kaufmann.
  11. Mirjalili, S. (2019). Genetic algorithm. In Evolutionary algorithms and neural networks (pp. 43-55). Springer, Cham.
  12. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.
  13. Asuncion, A., & Newman, D. (2007). UCI machine learning repository. Available: http://archive.ics.uci.edu/ml
  14. Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D., & Levine, A. J. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96(12), 6745-6750. https://doi.org/10.1073/pnas.96.12.6745
  15. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. the Journal of machine Learning research, 12, 2825-2830.