확률적 근사법과 공액기울기법을 이용한 다층신경망의 효율적인 학습

An Efficient Traning of Multilayer Neural Newtorks Using Stochastic Approximation and Conjugate Gradient Method

  • 조용현 (대구효성카톨릭대학교 전자.정보공학부)
  • 발행 : 1998.10.01

초록

본 논문에서는 신경망의 학습성능을 개선하기 위해 확룰적 근사법과 공액기울기법에 기초를 둔 새로운 학습방법을 제안하였다. 제안된 방법에서는 확률적 근사법과 공액기울기법을 조합 사용한 전역 최적화 기법의 역전파 알고리즘을 적용함으로써 학습성능을 최대한 개선할 수 있도록 하였다. 확률적 근사법은 국소최소점을 벗어나 전역최적점에 치우친 근사점을 결정해 주는 기능을 하도록 하며, 이점을 초기값으로 하여 결정론적 기법의 공액기울기법을 적용함으로써 빠른 수렴속도로 전역최적점으로의 수렴확률을 놓였다. 제안된 방법을 패리티 검사와 패턴 분류에 각각 적용하여 그 타당성과 성능을 확인한 결과 제안된 방법은 초기값을 무작위로 설정하는 기울기하강법에 기초를 둔 기존의 역전파 알고리즘이나 확률적 근사법과 기울기하강법에 기초를 둔 역전파 알고리즘에 비해 최적해로의 수렴 확률과 그 수렴속도가 우수함을 확인할 수 있었다.

This paper proposes an efficient learning algorithm for improving the training performance of the neural network. The proposed method improves the training performance by applying the backpropagation algorithm of a global optimization method which is a hybrid of a stochastic approximation and a conjugate gradient method. The approximate initial point for f a ~gtl obal optimization is estimated first by applying the stochastic approximation, and then the conjugate gradient method, which is the fast gradient descent method, is applied for a high speed optimization. The proposed method has been applied to the parity checking and the pattern classification, and the simulation results show that the performance of the proposed method is superior to those of the conventional backpropagation and the backpropagation algorithm which is a hyhrid of the stochastic approximation and steepest descent method.

키워드

참고문헌

  1. Parallel and Distributed Computation Numerical Method D.P.Bertsekas;J.N.Tsitsiklis
  2. Pattern Classification and Scene Analysis R.O.Duda;P.E.Hart
  3. Pattern Recognition Principles J.T.Tou;R.C.Gonzalez
  4. International Joint Conference on Neural Networks v.3 On the capacity of threelayernetworks S.Akaho;S.Amari
  5. handbook of Neural Computing Application A.J.Maren;C.T.Harston;R.M.Pap
  6. Neural Networks: Algorithms, Applications, and Programming Techniques J.A.Freeman;D.M.Skapura
  7. Parallel Distributed Processing D.E.Rumelhart;J.L.McClelland
  8. International Joint Conference on Neural Networks v.3 Learning the learning parameters R.Pedone;D.Parisi
  9. IEE., Proceeding-G v.139 no.3 Conjugate gradient algorithm for efficient traning of artificial neural networks C.Charalmbous
  10. Neural Networks v.4 no.1 Back-propagation algorithm which varies the number of hidden units Y.Hirose;K.Yamashita;S.Hijiya
  11. International Joint Conference on Neural Networks v.2 A self growing learning algorithm for determining the appropriate number of hidden units S.D.Wang;C.H.Hsu
  12. Neural Networks v.2 no.5 A new approach for finding the global minimum of error function of neural networks N.Baba
  13. Proceedings of International Joint Conference on Neural Networks v.1 A fast algorithm for finding global minima of error functions in layered neural networks J.Sun;W.I.Grosky;M.H.Hassoun
  14. JTC-CSCC Improving the training performances of the multilayer neural networks by SAS-based optimal estimation of initial weights Y.H.Cho;H.M.Choi
  15. Neural Networks v.3 no.4 Experiments in nonconvex optimization: stochastic approximation with function smoothing and simulated annealing M.A.Styblinski;T.S.Tang