Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지

Learning Ability of Deterministic Boltzmann Machine with Non-Monotonic Neurons in Hidden Layer

은닉층에 비단조 뉴런을 갖는 결정론적 볼츠만 머신의 학습능력에 관한 연구

  • 박철영 (대구대학교 정보통신공학부)
  • Published : 2001.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we evaluate the learning ability of non-monotonic DMM(Deterministic Boltzmann Machine) network through numerical simulations. The simulation results show that the proposed system has higher performance than monotonic DBM network model. Non-monotonic DBM network also show an interesting result that network itself adjusts the number of hidden layer neurons. DBM network can be realized with fewer components than other neural network models. These results enhance the utilization of non-monotonic neurons in the large scale integration of neuro-chips.

본 연구에서는 학습기근을 갖는 결정론적 볼츠만 머신의 은닉충 뉴런에 비단조 활성화 함수를 적요한 경위의 학습성능을 XOR 문제와 ADD 학습에 대하여 수지 시뮬레이션을 통하여분석한다. 단조 활성화함수를 사용한 경우와 비교하여 학습 수렴률, 학습안정도, 및 학습 속도에 있어서 성능이 크게 향상됨을 확인하였다. 또한 네트워크의 막전위 분포를 조사함으로서 end-cut-off 타입의 비단조 함수를 이용한 경우에 나타는 다음 층의 뉴런에 영향을 주지 않는 뉴런의 출현, 즉, 신경회로망에 있어서 은닉층 뉴런늬 수을 자율적으로 조정하는것을 확인하였따. 이것은 학습문제에 대하여 네트워크 은닉층 뉴런의 수를 명확하게 결정할수 없는 현재의 상황에 있어서는 새로운 돌파구가 될것으로 기대된다.

Keywords

References

  1. Neural Networks v.6 Memory search using complex dynamics in a recurrent neural network model S. Nara;P. Davis;H. Totsuji
  2. Neural Networks v.7 Oscillatory neural network and learning of continuously transformed patterns Y. Hayashi
  3. ICEICE Trans. on Fundamentals v.E79-A no.6 Limit cycles of one-dimensional neural networks with the cyclic connection matrix C. Y. Park;Y. Hayakawa;K. Nakajima;Y. Sawada
  4. IEICE Trans. Fundamentals v.E82-A no.6 Analog CMOS Implementation of Quantized Interconnection Neural Networks for Memorizing Limit Cycles C. Y. Park;K. Nakajima
  5. ICEICE Trans. on Fundamentals v.E82-A no.6 Majority Algorithm: A Formation for Neural Networks with the Quantized Connection Weights C. Y. Park;K. Nakajima
  6. IEICE Trans. v.J78-DⅡ Associative memory of sequential pattern using nonmonotone dynamics M. Morita
  7. Neural Networks v.9 Memory and Learning of Sequential Patterns by Nonmonotone Neural Networks M. Morita
  8. Proc. IEEE Int. Conf. Neural Networks v.3 A Theory on a Neural Net with Nonmonotone Neurons H. Yanai;S. Amari
  9. Neural Networks v.8 Retrival Properties of Analog Neural Networks and the Nonmonotonicity of Transfer Functions T. Fukai;J. Kim;M. Shiino
  10. Parallel distributed processing v.1 Learning and relearning in Boltzmann machines G. E. Hinton;T. J. Sejnowski
  11. Neural Computation v.1 Deterministic Boltzmann Learning Performs Steepest Descent in Weight Space G. E. Hinton
  12. Complex Systems v.1 A Mean Field Theory Learning Algorithm for Neural Networks C. Peterson;J. R. Anderson