Browse > Article
http://dx.doi.org/10.33851/JMIS.2021.8.2.101

Efficient Multi-scalable Network for Single Image Super Resolution  

Alao, Honnang (Department of Computer & Electronic Engineering, Sunmoon University)
Kim, Jin-Sung (Department of Computer & Electronic Engineering, Sunmoon University)
Kim, Tae Sung (Department of Computer & Electronic Engineering, Sunmoon University)
Lee, Kyujoong (Department of Computer & Electronic Engineering, Sunmoon University)
Publication Information
Journal of Multimedia Information System / v.8, no.2, 2021 , pp. 101-110 More about this Journal
Abstract
In computer vision, single-image super resolution has been an area of research for a significant period. Traditional techniques involve interpolation-based methods such as Nearest-neighbor, Bilinear, and Bicubic for image restoration. Although implementations of convolutional neural networks have provided outstanding results in recent years, efficiency and single model multi-scalability have been its challenges. Furthermore, previous works haven't placed enough emphasis on real-number scalability. Interpolation-based techniques, however, have no limit in terms of scalability as they are able to upscale images to any desired size. In this paper, we propose a convolutional neural network possessing the advantages of the interpolation-based techniques, which is also efficient, deeming it suitable in practical implementations. It consists of convolutional layers applied on the low-resolution space, post-up-sampling along the end hidden layers, and additional layers on high-resolution space. Up-sampling is applied on a multiple channeled feature map via bicubic interpolation using a single model. Experiments on architectural structure, layer reduction, and real-number scale training are executed with results proving efficient amongst multi-scale learning (including scale multi-path-learning) based models.
Keywords
Single-image Super Resolution; post-up-sampling; multi-scalable network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 H. Gao, H. Yuan, Z. Wang, and S. Ji, "Pixel transposed convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 42, no. 5, pp. 1-10, 2019.
2 Xiangxiang Chu, Bo Zhang, Hailong Ma, Ruijun Xu, Jixiang Li, and Qingyuan Li, "Fast, accurate and lightweight superresolution with neural architecture search," arXiv preprint arXiv:1901.07261, 2019.
3 J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image superresolution via sparse representation," IEEE Transactions on Image Processing (TIP), vol. 19, no. 11, pp. 2861-2873, 2010.   DOI
4 Huangzehao. Implementation of "Accurate Image Super-Resolution Using Very Deep Convolutional Networks". Github https://github.com/huangzehao/caffe-vdsr, 2017.
5 C. Dong, C. C. Loy, and X. Tang, "Accelerating the superresolution convolutional neural network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 1-17, 2016.
6 S. Gupta, P. P. Roy, D. P. Dogra, B.-G. Kim, "Retrieval of colour and texture images using local directional peak valley binary pattern," Pattern Analysis and Applications, vol. 23, no. 4, pp. 1569-1585, 2020.   DOI
7 Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and MingHsuan Yang "Deep laplacian pyramid networks for fast and accurate super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 624-632, 2017.
8 Armin Mehri, Parichehr B. Ardakani, and Angel D. Sappa, "Multi-Path Residual Network for Lightweight Image Super Resolution," in Proceeding of IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2704-2713, 2021.
9 Ding Liu, Zhaowen Wang, Yuchen Fan, Xianming Liu, ZhangyangWang, Shiyu Chang, and Thomas Huang, "Robust video super-resolution with learned temporal dynamics," in Proceedings of the IEEE International Conference on Computer Vision, pp. 2507-2515, 2017.
10 Wenzhe Shi, Jose Caballero, Christian Ledig, Xiahai Zhuang, Wenjia Bai, Kanwal Bhatia, Antonio M Simoes Monteiro de Marvao, Tim Dawes, Declan O'Regan, and Daniel Rueckert, "Cardiac image super-resolution with global correspondence using multi-atlas patchmatch," in Proceeding of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 9-16, 2013.
11 Wilman WW Zou and Pong C Yuen, "Very low-resolution face recognition problem," IEEE Transactions on image processing, vol. 21, no. 1, pp. 327-340, 2011.   DOI
12 Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646-1654, 2016.
13 C. Dong, C. C. Loy, K. He, and X. Tang, "Image superresolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 38, no. 2, pp, 295-307, 2015.   DOI
14 J. Kim, J. K. Lee, and K. M. Lee. Deeply-recursive convolutional network for image super-resolution," in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637-1645, 2016.
15 Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 136-144, 2017.
16 K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
17 S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502. 03167, 2015.
18 Y. Tai, J. Yang, X. Liu, C. Xu, "Memnet: A persistent memory network for image restoration," in Proceedings of the International Conference on Computer Vision (ICCV), pp. 4539-4547, 2017.
19 Roman Zeyde, Michael Elad, and Matan Protter "On single image scale-up using sparse-representations," in Proceeding of International conference on curves and surfaces, pp. 711-730, 2010.
20 Xiangyu He, Zitao Mo, Peisong Wang, Yang Liu, Mingyuan Yang, and Jian Cheng, "Ode-inspired network design for single image super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1732-1741, 2019.
21 P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 33, no. 5, pp. 898-916, 2011.   DOI
22 C. Dong, C. C. Loy, X. Tang, "Accelerating the super-resolution convolutional neural network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 392-407, 2016.
23 C. Ledig, L. Theis, F. Husz'ar, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang et al., "Photorealistic single image super-resolution using a generative adversarial network," in Proceeding of IEEE Computer Vision and Pattern Recognition, pp. 105-114, 2017.
24 Y. Fan, H. Shi, J. Yu, D. Liu, W. Han, H. Yu, Z. Wang, X. Wang, T. S. Huang, "Balanced two-stage residual networks for image super-resolution" in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 161-168, 2017.
25 S. Mukherjee, P. Kumar, R. Saini, P. P. Roy, D. P. Dogra, B. G. Kim, "Plant disease identification using deep neural networks," Journal of Multimedia Information System, vol. 4, no. 4, pp. 233-238, 2017.   DOI
26 A. K. Bhunia, A. K. Bhunia, A. Sain, P. P. Roy, "Improving document binarization via adversarial noise-texture augmentation," in Proceeding of 2019 IEEE International Conference on Image Processing (ICIP), pp. 2721-2725, 2019.
27 Y. Tai, J. Yang, X. Liu, "Image super-resolution via deep recursive residual network," in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2790-2798, 2017.
28 W. Shi, J. Caballero, F. Husz'ar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, Z. Wang, "Real-time single image and video super-resolution using an efficient sub-Fast, Accurate, and Lightweight Super-Resolution with CARN," arXiv:1803.08664v5, 2018.
29 N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 1-17, 2018.
30 Lukas Mino, Imrich Szabo, and Csaba Torok, "Bicubic splines and biquartic polynomials," Open Comput. Sci., vol. 6, pp. 1-7, 2016.   DOI
31 D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," CoRR, abs/1412.6980, pp. 1-15, 2014.
32 Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision (ECCV), pp. 184-199, 2014.
33 Pytorch deep learning framework, pytorch.org.
34 HolmesShuan. EDSR-ssim. github: https://github.com/HolmesShuan/EDSR-ssim, 2018.
35 Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel, "Low-complexity single- image super-resolution based on nonnegative neighbor embedding," in Proceedings of the British Machine Vision Conference (BMVC), pp. 1-10, 2012.