Browse > Article
http://dx.doi.org/10.6109/jkiice.2022.26.2.325

SoftMax Computation in CNN Using Input Maximum Value  

Kang, Hyeong-Ju (School of Computer Science and Engineering, Korea University of Technology and Education)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.26, no.2, 2022 , pp. 325-328 More about this Journal
Abstract
A convolutional neural network(CNN) is widely used in the computer vision tasks, but its computing power requirement needs a design of a special circuit. Most of the computations in a CNN can be implemented efficiently in a digital circuit, but the SoftMax layer has operations unsuitable for circuit implementation, which are exponential and logarithmic functions. This paper proposes a new method to integrate the exponential and logarithmic tables of the conventional circuits into a single table. The proposed structure accesses a look-up table (LUT) only with a few maximum values, and the LUT has the result value directly. Our proposed method significantly reduces the space complexity of the SoftMax layer circuit implementation. But our resulting circuit is comparable to the original baseline with small degradation in precision.
Keywords
CNN; SoftMax; Accelerator; Exponential function;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proceedings of Advances in Neural Information Processing Systems, Lake Tahoe: NV, pp. 1097-1105, 2012.
2 M. Tan and Q. V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in Proceedings of International Conference on Machine Learning, Long Beach: CA, pp. 6105-6114, 2019.
3 J. Park and D. Jeon, "Designing neuromorphic processor with on-chip learning," IDEC Journal of Integrated Circuits and Systems, vol. 6, no. 2, pp. 1-6, Apr. 2020.   DOI
4 Y. J. Kim and E. G. Kim, "Image based Fire Detection using Convolutional Neural Network," Journal of the Korea Institute of Information and Communication Engineering, vol. 20, no. 9, pp. 1649-1656, Sep. 2016.   DOI
5 K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proceedings of International Conference on Learning Representations, San Diego: CA, pp. 1-14, 2015.
6 J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Sog, S. Guadarrama, and K. Murphy, "Speed/accuracy trade-offs for modern convolutional object detectors," in Proceedings of Computer Vision and Pattern Recognition, Honolulu: HW, pp. 1-21, 2017.
7 M. Tan, R. Pang, and Q. V. Le, "EfficientDet: Scalable and Efficient Object Detection," in Proceedings of Computer Vision and Pattern Recognition, online, pp. 10781-10790, 2020.
8 B. Yuan, "Efficient hardware architecture of softmax layer in deep neural network," in Proceedings of IEEE International System-on-Chip Conference (SOCC), Seattle: WA, pp. 323-326, Sep. 2016.