Browse > Article
http://dx.doi.org/10.9717/kmms.2021.24.12.1598

A Study of Lightening SRGAN Using Knowledge Distillation  

Lee, Yeojin (Dept. of Electronic Engineering, Pukyong National University)
Park, Hanhoon (Dept. of Electronic Engineering, Pukyong National University)
Publication Information
Journal of Korea Multimedia Society / v.24, no.12, 2021 , pp. 1598-1605 More about this Journal
Abstract
Recently, convolutional neural networks (CNNs) have been widely used with excellent performance in various computer vision fields, including super-resolution (SR). However, CNN is computationally intensive and requires a lot of memory, making it difficult to apply to limited hardware resources such as mobile or Internet of Things devices. To solve these limitations, network lightening studies have been actively conducted to reduce the depth or size of pre-trained deep CNN models while maintaining their performance as much as possible. This paper aims to lighten the SR CNN model, SRGAN, using the knowledge distillation among network lightening technologies; thus, it proposes four techniques with different methods of transferring the knowledge of the teacher network to the student network and presents experiments to compare and analyze the performance of each technique. In our experimental results, it was confirmed through quantitative and qualitative evaluation indicators that student networks with knowledge transfer performed better than those without knowledge transfer, and among the four knowledge transfer techniques, the technique of conducting adversarial learning after transferring knowledge from the teacher generator to the student generator showed the best performance.
Keywords
CNN; Network Lightening; Knowledge Distillation; Super-Resolution; SRGAN;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. Blau, R. Mechrez, R. Timofte, T. Michaeli, and L. Zelnik-Manor, "The 2018 PIRM Challenge on Perceptual Image Super-Resolution," Proceeding of European Conference on Computer Vision, 2018.
2 Y. Zhang, T. Xiang, T.M. Hospedales, and H. Lu, "Deep Mutual Learning," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4320-4328, 2018.
3 I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.C. Courville, and Y. Bengio, "Generative Adversarial Networks," arXiv preprint, arXiv:1406.2661, 2014.
4 A. Romero, N. Ballas, S.E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, "FitNets: Hints for Thin Deep Nets," Proceeding of the International Conference on Learning Representations, 2015.
5 D. Lee, H.S. Lee, K. Lee, and H.-J. Lee, "Fast Very Deep Convolutional Neural Network with Deconvolution for Super-Resolution," Journal of Korea Multimedia Society, Vol. 20, No. 11, pp. 1750-1758, 2017.   DOI
6 G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," Proceeding of NIPS 2014 Deep Learning and Representation Learning Workshop, 2014.
7 C. Ledig, L. Theis, F. Huszar, J. Caballero, A.P. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, "Photo-Realistic Single Image Super- Resolution Using a Generative Adversarial Network," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 105-114, 2017.
8 E. Agustsson and R. Timofte, "Ntire 2017 Challenge on Single Image Super-Resolution: Dataset and Study," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 126-135, 2017.
9 L. Zhang, J. Song, A. Gao, J. Chen, C. Bao, and K. Ma, "Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation," Proceeding of International Conference on Computer Vision, pp. 3713-3722, 2019.
10 J. Gou, B. Yu, S.J. Maybank, and D. Tao "Knowledge Distillation: A Survey," International Journal of Computer Vision, Vol. 129, pp. 1789-1819, 2021.   DOI
11 N. Komodakis and S. Zagoruyko, "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer," Proceeding of the International Conference on Learning Representations, 2017.
12 X. Zhu, L. Zhang, L. Zhang, X. Liu, Y. Shen, and S. Zhao, "GAN-Based Image Super-Resolution with a Novel Quality Loss," Mathematical Problems in Engineering, Vol. 2020, Article ID 5217429, 2020.
13 Z. He, T. Dai, J. Lu, Y. Jiang, and S.-T. Xia, "Fakd: Feature-Affinity Based Knowledge Distillation for Efficient Image Super-Resolution," Proceeding of IEEE International Conference on Image Processing, pp. 518-522, 2020.
14 K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv preprint, arXiv:1409.1556, 2014.
15 X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C.C. Loy, "ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks," Proceeding of European Conference on Computer Vision, pp. 63-79, 2018.
16 H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms," IEEE Transactions on Image Processing, Vol. 15, No. 11, pp. 3440-3451, 2006.   DOI
17 C. Ma, C.Y. Yang, X. Yang, and M.-H. Yang, "Learning a No-Reference Quality Metric for Single-Image Super-Resolution," Computer Vision and Image Understanding, Vol. 158, pp. 1-16, 2017.   DOI
18 A. Mittal, R. Soundararajan, and A.C. Bovik, "Making a Completely Blind Image Quality Analyzer," IEEE Signal Processing Letters, Vol. 20, No. 3, pp. 209-212, 2013.   DOI