Journal of the Korean Institute of Telematics and Electronics B (전자공학회논문지B)
- Volume 33B Issue 4
- /
- Pages.209-216
- /
- 1996
- /
- 1016-135X(pISSN)
Learning of multi-layer perceptrons with 8-bit data precision
8비트 데이타 정밀도를 가지는 다층퍼셉트론의 역전파 학습 알고리즘
Abstract
In this paper, we propose a learning method of multi-layer perceptrons (MLPs) with 8-bit data precision. The suggested method uses the cross-entropy cost function to remove the slope term of error signal in output layer. To decrease the possibility of overflows, we use 16-bit weighted sum results into the 8-bit data with appropriate range. In the forwared propagation, the range for bit-conversion is determined using the saturation property of sigmoid function. In the backwared propagation, the range for bit-conversion is derived using the probability density function of back-propagated signal. In a simulation study to classify hadwritten digits in the CEDAR database, our method shows similar generalization performance to the error back-propagation learning with 16-bit precision.
Keywords