Effect of Training Sequence Control in On-line Learning for Multilayer Perceptron

다계층 퍼셉트론의 온라인 학습에서 학습 순서 제어의 효과

  • Received : 2010.04.22
  • Accepted : 2010.05.19
  • Published : 2010.07.15

Abstract

When human beings acquire and develop knowledge through education, their prior knowledge influences the next learning process. As this is a fact that should be considered in machine learning, we need to examine the effects of controlling the order of training sequence on machine learning. In this research, the role of the supervisor is extended to control the order of training samples, in addition to just instructing the target values for classification problems. The supervisor sequences the training examples categorized by SOM to the learning model which in this case is MLP. The proposed method is distinguished in that it selects the most instructive example from categories formed by SOM to assist the learning progress, while others use SOM only as a preprocessing method for training samples. The result shows that the method is effective in terms of the number of samples used and time taken in training.

인간이 교육을 통해 지식을 습득하고 발전시키는 과정에서, 이전 단계에서의 학습 진행 과정은 향후 학습에 영향을 미친다. 이것은 기계 학습에서도 고려되어야 할 사항으로 실제 기계 학습에서 학습순서의 제어가 어떤 효과가 있는지 살펴볼 필요가 있다. 본 연구에서는 MLP의 학습에서 지도자가 목표값을 알려주는 역할은 물론, 학습 대상의 지식 정도를 고려하여 자료들의 학습 순서를 제어하는 추가적 역할도 수행할 때, 학습 과정에 미치는 효과를 실험한다. 실험 방법은 SOM과 MLP를 이용하여 분류 문제에 적용한다. SOM은 지도자가 학습 순서를 결정하기 위한 학습 자료들의 범주화에 이용되고, MLP는 학습 대상이 된다. 제안하는 방법은 SOM을 학습 자료의 전처리 방법이 아닌, 학습 과정 동안 학습 자료의 선택에 이용하는 점에서 여타 연구들과 차이가 있으며, 실험 결과는 학습에 사용되는 자료의 수와 학습 횟수에서 개선 효과가 있음을 보여준다.

Keywords

References

  1. Simon Haykin, Neural Networks:A Comprehensive Foundation, 2nd Ed., Prentice Hall, 1999.
  2. MacKay, David, Information Theory, Inference and Learning Algorithms, Cambridge University Press. 2003.
  3. Bernard, I., "Multilayer perceptron and uppercase handwritten characters recognition," Proc. of the Second International Conference on Document Analysis and Recognition, pp.935-938, Oct. 1993.
  4. Markovic, M.Z., Milosavljevic, M.M., Samcovic, A.B., "A performance analysis of the multilayer perceptron in limited training data set conditions," Proc. 13th International Conference on Digital Signal Processing, vol.2, pp.719-722, Jul. 1997.
  5. Corinna Cortes, Vladimir Vapnik, "Support-Vector Networks," Machine Leaming, vol.20, pp.273-297, 1995.
  6. Xiaoguang Lu, Yunhong Wang, Jain, A.K., "Combining classifiers for face recognition," Proc. of International Conference on Multimedia and Expo, vol.3, pp.III-13-16, Jul. 2003.
  7. Rui Xu; Wunsch, D., "Survey of clustering algorithms," IEEE Transactions on Neural Networks, vol.16, no.3, pp.645-678, May. 2005. https://doi.org/10.1109/TNN.2005.845141
  8. Kohonen, T., "The self-organizing map," Proc. of the IEEE, vol.78, no.9, pp.1464-1480, Sept. 1990. https://doi.org/10.1109/5.58325
  9. Najet Arous, Noureddine Ellouze, "Cooperative supervised and unsupervised learning algorithm for phoneme recognition in continuous speech and speaker-independent context," Neurocomputing, vol.51, pp.225-235, Apr. 2003. https://doi.org/10.1016/S0925-2312(02)00618-5
  10. Hsieh, K.-R., Chen, W.-T., "A neural network model which combines unsupervised and supervised learning," IEEE Transactions on Neural Networks, vol.4, no.2, pp.357-360, Mar. 1993. https://doi.org/10.1109/72.207624
  11. Mingkun Li, Ishwar K.Sethi, "Confidence-Based Active Learning," IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.28, no.8, pp.1251-1261, Aug. 2006. https://doi.org/10.1109/TPAMI.2006.156
  12. Burr Settles, "Active Learning Literature Survey," Computer Sciences Technical Report 1648, University of Wisconsin-Madison. 2009.
  13. Olsson, Fredrik, "A literature survey of active machine learning in the context of natural language processing," SICS Technical Report T2009:06. Swedish Institute of Computer Science, Kista, Sweden. 2009.
  14. UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/