Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Successive Approximated Log Operation Circuit for SoftMax in CNN

CNN의 SoftMax 연산을 위한 연속 근사 방식의 로그 연산 회로

  • Kang, Hyeong-Ju (School of Computer Science and Engineering, Korea University of Technology and Education)
  • Received : 2020.12.13
  • Accepted : 2021.01.26
  • Published : 2021.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In a CNN for image classification, a SoftMax layer is usually placed at the end. The exponentinal and logarithmic operations in the SoftMax layer are not adequate to be implemented in an accelerator circuit. The operations are usually implemented with look-up tables, and the exponential operation can be implemented in an iterative method. This paper proposes a successive approximation method to calculate a logarithm to remove a very large look-up table. By substituing the large table with two very small tables, the circuit can be reduced much. The experimental results show that the 85% area reduction can be reached with a small error degradation.

Keywords

References

  1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proceedings of Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
  2. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proceedings of International Conference on Learning Representations, pp. 1-14, 2015.
  3. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of Computer Vision and Pattern Recognition, pp. 1-9, 2015.
  4. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of Computer Vision and Pattern Recognition, pp. 770-778, 2016.
  5. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single shot multibox detector," in Proceedings of European Conference on Computer Vision, pp. 21-37, 2016.
  6. Y.-J. Kim and E.-G. Kim, "Image based fire detection using convolutional neural network," Journal of the Korea Institute of Information and Communication Engineering, vol. 20, no. 9, pp. 1649-1656, Sep. 2016. https://doi.org/10.6109/jkiice.2016.20.9.1649
  7. B. Yuan, "Efficient hardware architecture of softmax layer in deep neural network," in Proceedings of IEEE International System-on-Chip Conference (SOCC), pp. 323-326, 2016.
  8. J. Park and D. Jeon, "Designing neuromorphic processor with on-chip learning," IDEC Journal of Integrated Circuits and Systems, vol. 6, no. 2, pp. 1-6, Apr. 2020.