Browse > Article
http://dx.doi.org/10.5370/KIEE.2018.67.12.1665

Evolutionary Computation Based CNN Filter Reduction  

Seo, Kisung (Department of Electronics Engineering, Seokyeong University)
Publication Information
The Transactions of The Korean Institute of Electrical Engineers / v.67, no.12, 2018 , pp. 1665-1670 More about this Journal
Abstract
A convolutional neural network (CNN), which is one of the deep learning models, has been very successful in a variety of computer vision tasks. Filters of a CNN are automatically generated, however, they can be further optimized since there exist the possibility of existing redundant and less important features. Therefore, the aim of this paper is a filter reduction to accelerate and compress CNN models. Evolutionary algorithms is adopted to remove the unnecessary filters in order to minimize the parameters of CNN networks while maintaining a good performance of classification. We demonstrate the proposed filter reduction methods performing experiments on CIFAR10 data based on the classification performance. The comparison for three approaches is analysed and the outlook for the potential next steps is suggested.
Keywords
Convolutional neural network; Filter reduction; Genetic algorithm;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Suganuma, s, Shirakawa, T. Nagao, "A Genetic Programming Approach to Designing Convolutional Neural Network Architectures", Proceedings of GECCO 2017, pp. 497-504, 2017
2 B. Zoph, and Q. V. Le, "Neural Architecture Search with Reinforcement Learning", CoRR abs/1611.01578 2016
3 C. Liu, B. Zoph, J. Shlens, W. Hua, L. J. Li, L. Fei-Fei, and K. Murphy, "Progressive Neural Architecture Search", ECCV 2018
4 E. Real, A. Aggarwal, Y. Huang, Q. V. Le, "Aging Evolution for Image Classifier Architecture Search", 2018
5 H. Hu, R. Peng, Y.W. Tai, and C.K. Tang. "Network trimming: A data-driven neuron pruning approach towards efficient deep architectures", arXiv preprint arXiv:1607.03250, 2016.
6 H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning Filters for Efficient ConvNets", CoRR abs/1608.08710 (2016).
7 Q. Huang, K. Zhou, S. You, and U. Neumann, "Learning to prune filters in convolutional neural networks", arXiv preprint arXiv:1801.07365. 2018 Jan 23.
8 C. Chen, F. Tung, N. Vedula, and G. Mori, "Constraint-Aware Deep Neural Network Compression", ECCV (8) 2018: 409-424.
9 S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding", arXiv preprint arXiv:1510.00149, 2015.
10 J. H. Luo, J. Wu, and W. Lin, "ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression", ICCV 2017: 5068-5076.
11 J. Kim, M. Lee, J. Choi, K. Seo, "GA-based Filter Selection for Representation in Convolutional Neural Networks", ECCV 2018 Workshop on Compact and Efficient Feature Representation and Learning in Computer Vision.
12 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition", Computer Vision and Pattern Recognition, 2016..
13 K. Seo, "Analysis of evolutionary optimization methods for CNN structures", Transactions of the Korean Institute of Electrical Engineers, 67(6), pp. 767-772, 2018   DOI
14 J. Schmidhuber, "Deep Learning in Neural Networks: An Overview", Neural Networks, Vol. 61, pp. 85-117, 2015.   DOI
15 Y. LeCun, Y. Bengio, G. Hinton, "Deep learning," Nature, Vol. 521, pp. 436-444, 2015.   DOI
16 LeCun, Yann, et al. "Gradient based learning applied to document recognition", Proceedings of the IEEE, pp. 2278-2324, 1998.
17 Yunwon Park, In So Kweon "Ambiguous Surface Defect Image Classification of AMOLEDD is playsin Smartphones", IEEE Trans. Industrial Informatics, 12(2): 597-607,2016   DOI
18 K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", International Conference on Learning Representations, 2014.
19 C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going Deeper with Convolutions", Computer Vision and Pattern Recognition, 2015
20 S. Zagoruyko and N. Komodakis, "Wide Residual Networks", arXiv: 1605.07146, 2016.
21 C. Fernando et al. "Convolution by Evolution: Differentiable Pattern Producing Networks", In Proceedings of the 2016 Genetic and Evolutionary Computation Conference, Denver, CO, USA, pp. 109-116. 2016.
22 A. Rikhtegar, M. Pooyan, M. Manzuri-Shalmani, "Genetic algorithm-optimised structure of convolutional neural network for face recognition applications", IET Computer Vision, Vol. 10, Iss. 6, pp. 559-566, 2016   DOI
23 L. Xie, A. Yuille, "Genetic CNN", CVPR 2017