• The KIPS Transactions:PartB
• /
• v.8B no.3
• /
• pp.237-242
• /
• 2001

고전적인 GLA 알고리즘과 마찬가지로 Kohonen 학습법은 경도 강하법으로 오차함수의 해에 접근해 나간다. 따라서 KLA의 이러한 문제를 극복하기 위해 모의 담금질법의 일종인 Cauchy 학습법을 응용을 제안한다. 그러나 이 방법은 학습시간이 느리다고 하는 단점이 있다. 본 논문 이 점을 개선시키기 위해 Cauchy 학습법과 Kohonen 학습법을 순차 결합시킨 또 다른 학습법을 제안한다. 그 결과 코시 학습법과 마찬가지로 국부최적 문제를 극복하면서도 삭습시간을 단축할 수 있었다.

• The Journal of the Acoustical Society of Korea
• /
• v.13 no.1
• /
• pp.5-13
• /
• 1994

This paper presents a study on Neural Networks for Phoneme Recognition and performs the Phoneme Recognition using TDNN (Time Delay Neural Network). Also, this paper proposes training algorithm for speech recognition using neural nets that is a proper to large scale TDNN. Because Phoneme Recognition is indispensable for continuous speech recognition, this paper uses TDNN to get accurate recognition result of phonemes. And this paper proposes new training algorithm that can converge TDNN to an optimal state regardless of the number of phonemes to be recognized. The recognition experiment was performed with new training algorithm for TDNN that combines backpropagation and Cauchy algorithm using stochastic approach. The results of the recognition experiment for three phoneme classes for two speakers show the recognition rates of $98.1\%$. And this paper yielded that the proposed algorithm is an efficient method for higher performance recognition and more reduced convergence time than TDNN.

• The Korean Journal of Applied Statistics
• /
• v.32 no.5
• /
• pp.693-702
• /
• 2019

Deep learning has gained popularity for the classification and prediction task. Neural network layers become deeper as more data becomes available. Saturation is the phenomenon that the gradient of an activation function gets closer to 0 and can happen when the value of weight is too big. Increased importance has been placed on the issue of saturation which limits the ability of weight to learn. To resolve this problem, Glorot and Bengio (Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 249-256, 2010) claimed that efficient neural network training is possible when data flows variously between layers. They argued that variance over the output of each layer and variance over input of each layer are equal. They proposed a method of initialization that the variance of the output of each layer and the variance of the input should be the same. In this paper, we propose a new method of establishing initialization by adopting truncated normal distribution and truncated cauchy distribution. We decide where to truncate the distribution while adapting the initialization method by Glorot and Bengio (2010). Variances are made over output and input equal that are then accomplished by setting variances equal to the variance of truncated distribution. It manipulates the distribution so that the initial values of weights would not grow so large and with values that simultaneously get close to zero. To compare the performance of our proposed method with existing methods, we conducted experiments on MNIST and CIFAR-10 data using DNN and CNN. Our proposed method outperformed existing methods in terms of accuracy.