Browse > Article

Dynamic Adjustment of the Pruning Threshold in Deep Compression  

Lee, Yeojin (Department of Electronic Engineering, Pukyong National University)
Park, Hanhoon (Department of Electronic Engineering, Pukyong National University)
Publication Information
Journal of the Institute of Convergence Signal Processing / v.22, no.3, 2021 , pp. 99-103 More about this Journal
Abstract
Recently, convolutional neural networks (CNNs) have been widely utilized due to their outstanding performance in various computer vision fields. However, due to their computational-intensive and high memory requirements, it is difficult to deploy CNNs on hardware platforms that have limited resources, such as mobile devices and IoT devices. To address these limitations, a neural network compression research is underway to reduce the size of neural networks while maintaining their performance. This paper proposes a CNN compression technique that dynamically adjusts the thresholds of pruning, one of the neural network compression techniques. Unlike the conventional pruning that experimentally or heuristically sets the thresholds that determine the weights to be pruned, the proposed technique can dynamically find the optimal thresholds that prevent accuracy degradation and output the light-weight neural network in less time. To validate the performance of the proposed technique, the LeNet was trained using the MNIST dataset and the light-weight LeNet could be automatically obtained 1.3 to 3 times faster without loss of accuracy.
Keywords
CNN; Neural network compression; Pruning; Threshold adjustment; LeNet;
Citations & Related Records
연도 인용수 순위
  • Reference
1 H. Park, "A review of 3D object tracking methods using deep learning," Journal of the Institute of Convergence Signal Processing, vol. 22, no. 1, pp. 30-37, 2021.
2 Y. J. Lee, Y. H. Moon, J. Y. Park, and O. G. Min, "Recent R&D trends for lightweight deep learning," Electronics and Telecommunications Trends, vol. 34, no. 2, pp. 40-50, 2019.
3 S. Han, H. Mao, and W. J. Dally, "Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding," Proc. of ICLR, 2016.
4 K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," Proc. of ICLR, 2015.
5 Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.   DOI
6 Deep-Compression-PyTorch,http://github.com/mightydeveloper/Deep-Compression-PyTorch.git
7 R. Yamashita, M. Nishio, R. Do, and K. Togashi, "Convolutional neural networks: an overview and application in radiology," Insights into Imaging, vol. 9, pp. 611-629, 2018.   DOI
8 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.   DOI
9 MNIST handwritten digit database, http://yann.lecun.com/exdb/mnist