Browse > Article
http://dx.doi.org/10.3837/tiis.2021.11.011

Lightweight Single Image Super-Resolution Convolution Neural Network in Portable Device  

Wang, Jin (School of Computer and Communication Engineering, Changsha University of Science and Technology)
Wu, Yiming (School of Computer and Communication Engineering, Changsha University of Science and Technology)
He, Shiming (School of Computer and Communication Engineering, Changsha University of Science and Technology)
Sharma, Pradip Kumar (Department of Computing Science, University of Aberdeen)
Yu, Xiaofeng (School of Business, Nanjing University)
Alfarraj, Osama (Computer Science Department, Community College, King Saud University)
Tolba, Amr (Computer Science Department, Community College, King Saud University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.11, 2021 , pp. 4065-4083 More about this Journal
Abstract
Super-resolution can improve the clarity of low-resolution (LR) images, which can increase the accuracy of high-level compute vision tasks. Portable devices have low computing power and storage performance. Large-scale neural network super-resolution methods are not suitable for portable devices. In order to save the computational cost and the number of parameters, Lightweight image processing method can improve the processing speed of portable devices. Therefore, we propose the Enhanced Information Multiple Distillation Network (EIMDN) to adapt lower delay and cost. The EIMDN takes feedback mechanism as the framework and obtains low level features through high level features. Further, we replace the feature extraction convolution operation in Information Multiple Distillation Block (IMDB), with Ghost module, and propose the Enhanced Information Multiple Distillation Block (EIMDB) to reduce the amount of calculation and the number of parameters. Finally, coordinate attention (CA) is used at the end of IMDB and EIMDB to enhance the important information extraction from Spaces and channels. Experimental results show that our proposed can achieve convergence faster with fewer parameters and computation, compared with other lightweight super-resolution methods. Under the condition of higher peak signal-to-noise ratio (PSNR) and higher structural similarity (SSIM), the performance of network reconstruction image texture and target contour is significantly improved.
Keywords
Deep learning; super-resolution; feedback mechanism; information distillation; coordinate attention;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," in Proc. of European Conference on Computer Vision, pp. 294-310, 2018.
2 N. Ahn, B. Kang, and K. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proc. of European Conference on Computer Vision, pp. 252-268, 2018.
3 W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Realtime single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874-1883, 2016.
4 K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, "GhostNet: more features from cheap operations," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1577-1586, 2020.
5 J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image superresolution," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637-1645, 2016.
6 H. Chen, Y. Wang, T. Guo, C. Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, and W. Gao, "Pre-trained image processing transformer," arXiv preprint arXiv:2012.00364, 2021.
7 Z. Hui, X. Wang, and X. Gao, "Fast and Accurate single image super-resolution via information distillation network," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 723-731, 2018.
8 Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, "Feedback network for image superresolution," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3862-3871, 2019.
9 E. Agustsson, and R. Timofte, "NTIRE 2017 challenge on single image super-resolution: dataset and study," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1122-1131, 2017.
10 B. Marco, R. Aline, G. Christine, and L. Marie, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," in Proc. of British Machine Vision Conference, pp. 135.1-135.10, 2012.
11 D. Zhang, T. Yin, G. Yang, M. Xia, L. Li, and X. Sun, "Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies," Journal of Visual Communication and Image Representation, vol. 48, no. 1, pp. 281-291, Feb. 2017.   DOI
12 J. Deng, W. Dong, R. Socher, L. Li, K. Li, and F. Li, "ImageNet: a large-Scale hierarchical image database," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
13 A. G. Howard, M. Zhu, B. Chen, D. Kalenicheko, W. Wang, T. Weyand, M. Andreeto, and H. Adam, "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications," arXiv preprint arXiv:1704.04861, 2017.
14 D. Zhou, Y. Chen, J. Feng, and S. Yan, "Rethinking Bottleneck Structure for Efficient Mobile Network Design," in Proc. of European Conference on Computer Vision, pp. 680-697, 2020.
15 M. Long, F. Peng, and Y. Zhu, "Identifying natural images and computer-generated graphics based on binary similarity measures of PRNU," Multimedia Tools and Applications, vol. 78, no. 1, pp. 489-506, May. 2019.   DOI
16 Y. Gui, and G. Zeng, "Joint learning of visual and spatial features for edit propagation from a single image," Visual Computer, vol. 36, no. 3, pp. 469-482, Jun. 2020.   DOI
17 X. Zhang, F. Peng, and M. Long, "Robust coverless image steganography based on DCT and LDA topic classification," IEEE Transactions on Multimedia, vol. 20, no. 12, pp. 3223-3238, Oct. 2018.   DOI
18 J. Wang, Y. Wu, L. W, L. W, O. Alfarraj, and A. Tolba, "Lightweight feedback convolution neural network for remote sensing images super-resolution," IEEE Access, vol. 9, no. 1, pp. 15992-16003, Feb. 2021.   DOI
19 C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, Nov. 2016.   DOI
20 K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: surpassing human-level performance on imagenet classification," in Proc. of IEEE International Conference on Computer Vision, pp. 1026-1034, 2015.
21 W. S. Lai, J. B. Huang, N. Ahuja, and M. H. Yang, "Deep laplacian pyramid networks for fast and accurate super resolution," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5835-5843, 2017.
22 Z. Hui, X. Gao, Y. Yang, and X. Wang, "Lightweight image super-resolution with information multi-distillation network," in Proc. of ACM the 27th International Conference on Multimedia, pp. 2024-2032, 2019.
23 A. R. Zamir, W. Wu, L. Sun, W. B. Shen, and B. E. Shi, "Feedback networks," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1808-1817, 2017.
24 Q. Hou, D. Zhou, and J. Feng, "Coordinate attention for efficient mobile network design," arXiv preprint arXiv:2103.02907, 2021.
25 Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual Network," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790-2798, 2017.
26 Y. Tai, J. Yang, X. Liu, and C. Xu, "Memnet: A persistent memory network for image restoration," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4539-4547, 2017.
27 K. Zhang, W. Zuo, and L. Zhang, "Learning a single convolutional super-resolution network for multiple degradations," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262-3271, 2018.
28 J. Kim, J. Choi, M. Cheon, and J. Lee, "MAMNet: Multi-path Adaptive Modulation Network for Image Super-Resolution," arXiv preprint arXiv:1811.12043, 2018.
29 B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image super-resolution," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1132-1140, 2017.
30 C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in Proc. of European Conference on Computer Vision, pp. 391-407, 2016.
31 Z. Roman, E. Michael, and P. Matan, "On single image scale-up using sparse-representations," in Proc. of Springer Curves and Surfaces, pp. 711-730, 2010.
32 D. Martin, C. Fowlkes, D. Tal, J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. of IEEE International Conference on Computer Vision, pp. 416-425, 2001.
33 J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646-1654, 2016.
34 J. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed selfexemplars," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp.5197-5206, 2015.
35 Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki, and K. Aizawa, "Sketch-based manga retrieval using manga109 dataset," Multimedia Tools and Applications, vol. 76, no. 20, pp. 21811-21838, Nov. 2017.   DOI
36 D. P. Kingma, and B. Jimmy, "Adam: a method for stochastic optimization," in Proc. of IEEE International Conference on Learning Representations, 2015.
37 D. Song, C. Xu, X. Jia, Y. Chen, C. Xu, and Y. Wang, "Efficient residual dense block search for image super-resolution," in Proc. of Educational Advances in Artificial Intelligence, vol. 34, no. 7, pp. 12007-12014, 2020.
38 A. Vaswani, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," in Proc. of Neural Information Processing Systems, pp. 5998-6008, 2017.