Browse > Article
http://dx.doi.org/10.3745/JIPS.02.0168

Lightweight Single Image Super-Resolution by Channel Split Residual Convolution  

Liu, Buzhong (School of Electronic Network, Jiangsu Vocational College of Electronics and Information)
Publication Information
Journal of Information Processing Systems / v.18, no.1, 2022 , pp. 12-25 More about this Journal
Abstract
In recent years, deep convolutional neural networks have made significant progress in the research of single image super-resolution. However, it is difficult to be applied in practical computing terminals or embedded devices due to a large number of parameters and computational effort. To balance these problems, we propose CSRNet, a lightweight neural network based on channel split residual learning structure, to reconstruct highresolution images from low-resolution images. Lightweight refers to designing a neural network with fewer parameters and a simplified structure for lower memory consumption and faster inference speed. At the same time, it is ensured that the performance of recovering high-resolution images is not degraded. In CSRNet, we reduce the parameters and computation by channel split residual learning. Simultaneously, we propose a double-upsampling network structure to improve the performance of the lightweight super-resolution network and make it easy to train. Finally, we propose a new evaluation metric for the lightweight approaches named 100_FPS. Experiments show that our proposed CSRNet not only speeds up the inference of the neural network and reduces memory consumption, but also performs well on single image super-resolution.
Keywords
Channel Split Residual; Double-Upsampling; Lightweight; Super-Resolution;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Zhang, X. Zhou, M. Lin, and J. Sun, "Shufflenet: an extremely efficient convolutional neural network for mobile devices," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 6848-6856.
2 D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proceedings 8th IEEE International Conference on Computer Vision (ICCV), Vancouver, Canada, 2001, pp. 416-423.
3 J. B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, 2015, pp. 5197-5206.
4 C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 105-114.
5 W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 2016, pp. 1874-1883.
6 C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in Computer Vision - ECCV 2016. Cham, Switzerland: Springer, 2016, pp. 391-407.
7 Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 2709-2798.
8 N. Ahn, B. Kang, and K. A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018, pp. 256-272.
9 X. Chu, B. Zhang, R. Xu, and H. Ma, "Multi-objective reinforced evolution in mobile neural architecture search," 2019 [Online]. Available: https://arxiv.org/abs/1901.01074.
10 X. Chu, B. Zhang, H. Ma, R. Xu, and Q. Li, "Fast, accurate and lightweight super-resolution with neural architecture search," 2019 [Online]. Available: https://arxiv.org/abs/1901.07261.
11 N. Ahn, B. Kang, and K. A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018, pp. 256-272.
12 Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 2472-2481.
13 S. Nah, T. H. Kim, and K. M. Lee, "Deep multi-scale convolutional neural network for dynamic scene deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 257-265.
14 R. Lan, L. Sun, Z. Liu, H. Lu, C. Pang, and X. Luo, "MADNet: a fast and lightweight network for singleimage super resolution," IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1443-1453, 2020.
15 K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 2016, pp. 770-778.
16 Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," in Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018, pp. 294-310.
17 M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Low-complexity single-image superresolution based on nonnegative neighbor embedding," in Proceedings of the British Machine Vision Conference (BMVC), Surrey, UK, 2012, pp. 1-10.
18 Z. Hui, X. Wang, and X. Gao, "Fast and accurate single image super-resolution via information distillation network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 723-731.
19 C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in Computer Vision - ECCV 2014. Cham, Switzerland: Springer, 2014, pp. 184-199.
20 J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 2016, pp. 1637-1645.
21 B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, "Learning transferable architectures for scalable image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 8697-8710.
22 B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced deep residual networks for single image superresolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, 2017, pp. 1132-1140.
23 B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image superresolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, 2017, pp. 1132-1140.
24 A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, "Mobilenets: efficient convolutional neural networks for mobile vision applications," 2017 [Online]. Available: https://arxiv.org/abs/1704.04861.
25 Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, "Feedback network for image super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, 2019, pp. 3867-3876.
26 K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, "Ghostnet: more features from cheap operations," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, 2020, pp. 1577-1586.
27 D. P. Kingma and J. Ba, "Adam: a method for stochastic optimization," 2014 [Online]. Available: https://arxiv.org/abs/1412.6980.
28 L. Zhang, P. Wang, C. Shen, L. Liu, W. Wei, Y. Zhang, and A. Van Den Hengel, "Adaptive importance learning for improving lightweight image super-resolution network," International Journal of Computer Vision, vol. 128, no. 2, pp. 479-499, 2020.   DOI
29 R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Curves and Surfaces. Heidelberg, Germany: Springer, 2010, pp. 711-730.
30 E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: dataset and study," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, 2017, pp. 1122-1131.
31 Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, "Feedback network for image super-resolution," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, 2019, pp. 3867-3876.
32 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.   DOI