Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Efficient Incremental Learning using the Preordered Training Data

미리 순서가 매겨진 학습 데이타를 이용한 효과적인 증가학습

  • 이선영 (포항공과대학교 컴퓨터공학과) ;
  • 방승양 (포항공과대학교 컴퓨터공학과)
  • Published : 2000.02.15
Download PDF
⟨ Previous Next ⟩

Abstract

Incremental learning generally reduces training time and increases the generalization of a neural network by selecting training data incrementally during the training. However, the existing methods of incremental learning repeatedly evaluate the importance of training data every time they select additional data. In this paper, an incremental learning algorithm is proposed for pattern classification problems. It evaluates the importance of each piece of data only once before starting the training. The importance of the data depends on how close they are to the decision boundary. The current paper presents an algorithm which orders the data according to their distance to the decision boundary by using clustering. Experimental results of two artificial and real world classification problems show that this proposed incremental learning method significantly reduces the size of the training set without decreasing generalization performance.

증가학습은 점진적으로 학습 데이타를 늘려가며 신경망을 학습시킴으로써 일반적으로 학습시간을 단축시킬 뿐만 아니라 신경망의 일반화 성능을 향상시킨다. 그러나, 기존의 증가학습은 학습 데이타를 선정하는 과정에서 데이타의 중요도를 반복적으로 평가한다. 본 논문에서는 분류 문제의 경우 학습이 시작되기 전에 데이타의 중요도를 한 번만 평가한다. 제안된 방법에서는 분류 문제의 경우 클래스 경계에 가까운 데이타일수록 그 데이타의 중요도가 높다고 보고 이러한 데이타를 선택하는 방법을 제시한다. 두가지 합성 데이타와 실세계 데이타의 실험을 통해 제안된 방법이 기존의 방법보다 학습 시간을 단축시키며 일반화 성능을 향상시킴을 보인다.

Keywords

References

  1. D. E. Rumelhart, G. E. Hinton, and R. J. Williams,'Learning Internal Representations by Error Propation,' Parallel Distributed Processing, Vol. 1, pp. 318-362, Bradford Books, Cambridge MA, 1986
  2. C. Cachin, Pedagogical Pattern Selection Strategies, Neural Networks, Vol. 7, No. 1, pp. 175-181, 1994 https://doi.org/10.1016/0893-6080(94)90066-3
  3. S. Ahmad and G. Tesauro, 'Scaling and generalization in neural networks : a case study,' in Proc. 1988 Connec. Models Summer School, pp. 3-10, 1989
  4. K. A. Huyser and M. A. Horowitz, 'Generalization in connectionist networks that realize Boolean functions,' in Proc. 1988 Connec. Models Summer School, pp. 191- 200, 1989
  5. C. Cachin, Pedagogical Pattern Selection Strategies, Neural Networks, Vol. 7, No. 1, pp. 175-181, 1994 https://doi.org/10.1016/0893-6080(94)90066-3
  6. I. Cloete, J. Ludik. 'Increased Complexity Training,' IWANN, pp. 267-271, 1993
  7. J. Ludik, I. Cloete, 'Incremental Increased Complexity Training,' ESANN, Brussels, Belgium, pp. 161-165, 1994
  8. B. T. Zhang, Accelerated Learning by Active Example Selection, International Journal of Neural Systems, Vol. 5, No. 1, pp. 67-75, 1994
  9. A. Robel, The dynamic pattern selection algorithm: Effective training and controlled generalization of backpropagation neural networks, Technische Universitat Berlin, Germany, Tech. Rep., 1994
  10. D. A. Cohn, 'Neural Network Exploration using Optimal Experiment Design,' AI Memo 1491, MIT Artificial Intelligence Laboratory, 1994
  11. M. Plutowski, H. White, 'Selecting Concise Training Sets from Clean Data,' IEEE Transactions on Neural Netowrks, Vol. 4, No. 2, pp. 305-318, 1993 https://doi.org/10.1109/72.207618
  12. K. K. Sung, P. Niyogi, 'A Formulation for Active Learning with Applications to Object Detection, AI Memo 1438, MIT Artificial Intelligence Laboratory, 1996
  13. A. Englbrecht, I. Cloete, Selective Learning using Sensitivity Analysis, IJCNN98, Vol.2, pp.1150-1155, 1998 https://doi.org/10.1109/IJCNN.1998.685935
  14. R. Lange, R. Manner, 'Quantifying a Critical Training Set Size for Generalization and Overfitting using Teacher Neural Network' ICANN, Vol 1, pp. 497-500. 1994
  15. D. Cohn, L. Altlas, R, Ladner, Improving Generalization with Active Learning. Machine Learning Vol. 15 pp. 201-221. 1994 https://doi.org/10.1007/BF00993277
  16. Chulee Lee, David A. Landgrebe, 'Decision boundary Feature Extraction for Neural Networks' IEEE Transactions on Neural Nwtwork,' Vol. 8, No. 1 pp. 75-83, 1997 https://doi.org/10.1109/72.554193
  17. A. K. Jain and R. C. Dubges, 'Algorithm for Clustering Data,' Englewood Cliffs, NJ07632 : Prentice Hall, 1988
  18. http://ftp.ics.uci.edu/pub/machine-learning-databases/optdigits/