Browse > Article

An Incremental Rule Extraction Algorithm Based on Recursive Partition Averaging  

Han, Jin-Chul (명지대학교 컴퓨터공학과)
Kim, Sang-Kwi (명지대학교 컴퓨터공학과)
Yoon, Chung-Hwa (명지대학교 컴퓨터공학과)
Publication Information
Journal of KIISE:Software and Applications / v.34, no.1, 2007 , pp. 11-17 More about this Journal
Abstract
One of the popular methods used for pattern classification is the MBR (Memory-Based Reasoning) algorithm. Since it simply computes distances between a test pattern and training patterns or hyperplanes stored in memory, and then assigns the class of the nearest training pattern, it cannot explain how the classification result is obtained. In order to overcome this problem, we propose an incremental teaming algorithm based on RPA (Recursive Partition Averaging) to extract IF-THEN rules that describe regularities inherent in training patterns. But rules generated by RPA eventually show an overfitting phenomenon, because they depend too strongly on the details of given training patterns. Also RPA produces more number of rules than necessary, due to over-partitioning of the pattern space. Consequently, we present the IREA (Incremental Rule Extraction Algorithm) that overcomes overfitting problem by removing useless conditions from rules and reduces the number of rules at the same time. We verify the performance of proposed algorithm using benchmark data sets from UCI Machine Learning Repository.
Keywords
Rule Extraction; Overfitting Problem; Incremental Learning Algorithm;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Ian H. Witten, Eibe Frank, Data Mining, Morgan Kaufmann, 1999
2 Rissanen, J., 'Modelling by shortest data description,' Automatica, 14, pp.45-471, 1978
3 이형일, 정태선, 윤충화, 강경식, '재귀분할 평균법을 이용한 새로운 메모리기반 추론 알고리즘,' 한국정보처리학회 논문지, Vol.6, No.7, pp.1849-1857, 1999
4 Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J., 'UCI Repository of machine learning databases,' Irvine, CA: University of California, Department of Information and Computer Science, 1998 [http:// www.ics.uci.edu/～mlearn/MLRepository.html]
5 T. Dietterich, 'A Study of Distance-Based Machine Learning Algorithms,' Ph. D. Thesis, computer Science Dept., Oregon State University, 1995
6 Cohen, W. W., 'Fast effective rule induction,' In Proceedings of the 12th International Conference on Machine Learning, pp. 115-123, Morgan Kaufmann, 1955
7 D. Aha, 'A Study of Instance-Based Algorithms for Supervised Learning Tasks: Mathematical, Empirical, and psychological Evaluations,' Ph. D. Thesis, Information and Computer Science Dept., University of California, Irvine, 1990
8 J. R. Quinlan,' Induction of decision trees,' Machine Learning, Vol.1, No.1, pp.81-106, 1986
9 Eibe Frank, Ian H. Witten, 'Generating accurate rule sets without global optimization,' Proc. 15th International Conference on Machine Learning, pp. 144-151, Morgan Kaufmann, San Francisco, CA, 1998
10 J. R. Quinlan, 'C4.5: Programs for Machine Learning,' Morgan Kaufmann Publishers, San Mateo, CA, 1993
11 J. Cendrowska, 'PRISM : An Algorithm for inducing modular rules,' International Journal of Man-Machine Studies, 27(4): pp.349-370, 1987   DOI   ScienceOn
12 Johannes Fürnkranz, Gerhard Widmer, 'Incremental Reduced Error Pruning,' Proceedings of the 11th International Conference on Machine Learning, Morgan Kaufmann, pp.70-77, 1994
13 J. R. Quinlan, 'Simplifying Decision Trees,' Knowledge Acquisition for Knowledge-Based Systems, pp.239-252, Academic Press, 1988