Application and Performance Analysis of Double Pruning Method for Deep Neural Networks
![]() |
Lee, Seon-Woo
(Electric Computer Engineering, Inha University)
Yang, Ho-Jun (Computer Engineering, Inha University) Oh, Seung-Yeon (Computer Engineering, Inha University) Lee, Mun-Hyung (Computer Engineering, Inha University) Kwon, Jang-Woo (Computer Engineering, Inha University) |
1 | Z. Liu, M. Sun, T. Zhou, G. Huang, T. Darrell. (2019). Rethinking the Value of Network Pruning, International Conference on Learning Representations (ICLR) Seq |
2 | A. Morcos, H. Yu, M. Paganini & Y. Tian. (2019). One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. In Advances in Neural Information Processing Systems (pp. 4932-4942). |
3 | A. Krizhevsky & Hinton, G. (2009). Learning multiple layers of features from tiny images. |
4 | I. J. Goodfellow et al. (2013, Nov). Challenges in Representation Learning: A report on three machine learning contests. In International Conference on Neural Information Processing (pp. 117-124). Springer, Berlin, Heidelberg. |
5 | E. Barsoum, C. Zhang, C. C. Ferrer & Z. Zhang. (2016, October). Training deep networks for facial expression recognition with crowd-sourced label distribution. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 279-283). |
6 | A. Wong. (2019, August). NetScore: Towards universal metrics for large-scale performance analysis of deep neural networks for practical on-device edge usage. In International Conference on Image Analysis and Recognition (pp. 15-26). Springer, Cham. |
7 | M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard & Q. V. Le. (2019). Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2820-2828). |
8 | P. Ramachandran, B. Zoph & Q. V. Le. (2017). Searching for activation functions. arXiv preprint arXiv:1710.05941. |
9 | I. Loshchilov & F. Hutter. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983. |
10 | A. Aimar et al. (2018). Nullhop: A flexible convolutional neural network accelerator based on sparse representations of feature maps. IEEE transactions on neural networks and learning systems, 30(3), 644-656. DOI : 10.1109/TNNLS.2018.2852335 DOI |
11 | M. Schmidt, G. Fung & R. Rosales. (2007, September). Fast optimization methods for l1 regularization: A comparative study and two new approaches. In European Conference on Machine Learning (pp. 286-297). Springer, Berlin, Heidelberg. DOI : 10.1007/978-3-540-74958-5_28 |
12 | M. Sandler, A. Howard, M. Zhu, A. Zhmoginov & L. C. Chen. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510-4520). |
13 | E. Real, A. Aggarwal, Y. Huang & Q. V. Le. (2019, July). Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, 33, 4780-4789. DOI : 10.1609/aaai.v33i01.33014780 |
14 | S. Karen & Z. Andrew. (2014), Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv preprint arXiv:1409.1556. |
15 | K. He, X. Zhang, S. Ren & J. Sun. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). |
16 | C. Szegedy et al. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9). |
17 | A. Howard. et al. (2019). Searching for mobilenetv3. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1314-1324). |
18 | G. H. Andrew, Z. Menglong, C. Bo, K. Dmitry, W. Weijun, W. Tobias, A. Marco & A. Hartwig. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv preprint arXiv:1704.04861. |
19 | N. I, Forrest. H. Song, W. M. Matthew, A. Khalid, J. D. William & K. Kurt. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. arXiv preprint arXiv:1602.07360. |
20 | X. Zhang, X. Zhou, M. Lin & J. Sun. (2018). Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6848-6856). |
21 | F. Chollet. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258). |
22 | H. Song, M. Huizi & J. D. William (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149. |
23 | Gholami, A., Kwon K., Wu B., Tai Z., Yue X., Jin P., Zhao S., Keutzer K., (2018. June). Squeezenext: Hardware-aware neural network design. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 1638-1647). |
24 | L. Yann, S. D. John & A. S. Sara. (1990). Optimal brain damage. In Advances in neural information processing systems (pp. 598-605). |
25 | S. Han, J. Pool, J. Tran & W. Dally. (2015). Learning both weights and connections for efficient neural network. In Advances in neural information processing systems (pp. 1135-1143). |
26 | R. Reed, (1993). Pruning algorithms-a survey. IEEE transactions on Neural Networks, 4(5), 740-747. DOI : 10.1109/72.248452 DOI |
27 | N. Lee, T. Ajanthan & P. H. Torr. (2018). Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340. |
28 | Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan & C. Zhang, (2017). Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2736-2744). |
29 | Y. He, X. Zhang & J. Sun, (2017). Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1389-1397). |
30 | M. Tan & Q. V. Le. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946. |
31 | M. Tan & Q. V. Le. (2019). Mixconv: Mixed depthwise convolutional kernels. CoRR, abs/1907.09595 |
32 | J. H. Luo, J. Wu & W. Lin. (2017). Thinet: A filter level pruning method for deep neural network compression. The IEEE International Conference on Computer Vision (ICCV) (pp. 5058-5066) |
![]() |