Browse > Article
http://dx.doi.org/10.33851/JMIS.2021.8.4.203

Interpolation based Single-path Sub-pixel Convolution for Super-Resolution Multi-Scale Networks  

Alao, Honnang (Department of Computer & Electronic Engineering, Sunmoon University)
Kim, Jin-Sung (Department of Computer & Electronic Engineering, Sunmoon University)
Kim, Tae Sung (Department of Computer & Electronic Engineering, Sunmoon University)
Oh, Juhyen (Department of Computer & Electronic Engineering, Sunmoon University)
Lee, Kyujoong (Department of Computer & Electronic Engineering, Sunmoon University)
Publication Information
Journal of Multimedia Information System / v.8, no.4, 2021 , pp. 203-210 More about this Journal
Abstract
Deep leaning convolutional neural networks (CNN) have successfully been applied to image super-resolution (SR). Despite their great performances, SR techniques tend to focus on a certain upscale factor when training a particular model. Algorithms for single model multi-scale networks can easily be constructed if images are upscaled prior to input, but sub-pixel convolution upsampling works differently for each scale factor. Recent SR methods employ multi-scale and multi-path learning as a solution. However, this causes unshared parameters and unbalanced parameter distribution across various scale factors. We present a multi-scale single-path upsample module as a solution by exploiting the advantages of sub-pixel convolution and interpolation algorithms. The proposed model employs sub-pixel convolution for the highest scale factor among the learning upscale factors, and then utilize 1-dimension interpolation, compressing the learned features on the channel axis to match the desired output image size. Experiments are performed for the single-path upsample module, and compared to the multi-path upsample module. Based on the experimental results, the proposed algorithm reduces the upsample module's parameters by 24% and presents slightly to better performance compared to the previous algorithm.
Keywords
Super-resolution; multi-scalable network; multi-scale single-path Super-resolution;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. Ledig, L. Theis, F. Husz'ar, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang et al., "Photorealistic single image super-resolution using a generative adversarial network," in Proceeding of IEEE Computer Vision and Pattern Recognition, pp. 105-114, 2017.
2 JH Kim, BG Kim, PP Roy, and DM Jeong, "Efficient facial expression recognition algorithm based on hierarchical deep neural network structure," in Proceeding of IEEE access 7, 41273-41285, 2019.   DOI
3 Honnang Alao, Jin-Sung Kim, Tae Sung Kim, and Kyujoong Lee. "Efficient multi-scalable network for single-image super-resolution," in Journal of Multimedia Information System, Volume, No. 2, pp. 1-10, 2021.   DOI
4 Young-Hyun Jun, Jong-Ho Yun, and Myung-Ryul Choi. "Modified Cubic Convolution Interpolation for Low Computational Complexity," in the Korean Information Display Society, pp. 1259-1261, 2006.
5 Radu Timofte, Eirikur Agustsson, Luc Van Gool, MingHsuan Yang, Lei Zhang, Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, Kyoung Mu Lee, et al. "NTIRE 2017 challenge on single image super-resolution: Methods and results", in CVPR Workshops, pp. 1110-1121. IEEE Computer Society, 2017.
6 Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision (ECCV), pp. 184-199, 2014.
7 Roman Zeyde, Michael Elad, and Matan Protter "On single image scale-up using sparse-representations," in Proceeding of International conference on curves and surfaces, pp. 711-730, 2010.
8 J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Transactions on Image Processing (TIP), vol. 19, no. 11, pp. 2861-2873, 2010.   DOI
9 D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," CoRR, abs/1412.6980, pp. 1-15, 2014.
10 Sewar python library for image quality metrics: https://sewar.readthedocs.io/en/latest/
11 Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. "Image super-resolution using very deep residual channel attention networks", in ECCV (7), volume 11211 of Lecture Notes in Computer Science, pp. 294-310, 2018.
12 Xiangxiang Chu, Bo Zhang, Hailong Ma, Ruijun Xu, Jixiang Li, and Qingyuan Li, "Fast, accurate and lightweight super resolution with neural architecture search," arXiv preprint arXiv:1901.07261, 2019.
13 N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 1-17, 2018.
14 Armin Mehri, Parichehr B. Ardakani, and Angel D. Sappa, "Multi-Path Residual Network for Lightweight Image Super Resolution," in Proceeding of IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2704-2713, 2021.
15 J. Kim, J. K. Lee, and K. M. Lee. "Deeply-recursive convolutional network for image super-resolution," in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637-1645, 2016.
16 Wilman WW Zou and Pong C Yuen, "Very lowresolution face recognition problem," IEEE Transactions on image processing, vol. 21, no. 1, pp. 327-340, 2011.   DOI
17 Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," in Proceedings of the British Machine Vision Conference (BMVC), pp. 1-10, 2012.
18 Ding Liu, Zhaowen Wang, Yuchen Fan, Xianming Liu, ZhangyangWang, Shiyu Chang, and Thomas Huang, "Robust video super-resolution with learned temporal dynamics," in Proceedings of the IEEE International Conference on Computer Vision, pp. 2507-2515, 2017.
19 HolmesShuan.EDSR-ssim.github: https://github.com/ HolmesShuan/EDSR-ssim, 2018.
20 Wenzhe Shi, Jose Caballero, Christian Ledig, Xiahai Zhuang, Wenjia Bai, Kanwal Bhatia, Antonio M Simoes Monteiro de Marvao, Tim Dawes, Declan O'Regan, and Daniel Rueckert, "Cardiac image super-resolution with global correspondence using multi-atlas patch match," in Proceeding of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 9-16, 2013.
21 C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 38, no. 2, pp, 295-307, 2015.   DOI
22 C. Dong, C. C. Loy, X. Tang, "Accelerating the super-resolution convolutional neural network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 392-407, 2016.
23 Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 136-144, 2017.
24 Y. Fan, H. Shi, J. Yu, D. Liu, W. Han, H. Yu, Z. Wang, X. Wang, T. S. Huang, "Balanced two-stage residual networks for image super-resolution" in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 161-168, 2017.
25 W. Shi, J. Caballero, F. Husz'ar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, Z. Wang, "Real-time single image and video super-resolution using an efficient sub-Fast, Accurate, and Lightweight Super-Resolution with CARN," arXiv: 1803.08664v5, 2018.
26 A Mittal, P. P. Roy, P. Singh, and B. Raman. "Rotation and script independent text detection from video frames using sub pixel mapping," in Journal of Visual Communication and Image Representation, Volume No. 46, pp. 187-19, 2017.   DOI
27 Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646-1654, 2016.