Journal of Internet Computing and Services (인터넷정보학회논문지)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Study of Efficient Network Structure for Real-time Image Super-Resolution

실시간 영상 초해상도 복원을 위한 효율적인 신경망 구조 연구

  • Jeong, Woojin (Department of computer science and engineering, Hanyang University) ;
  • Han, Bok Gyu (Department of computer science and engineering, Hanyang University) ;
  • Lee, Dong Seok (Image PGM Team, Hanwha Systems) ;
  • Choi, Byung In (Image PGM Team, Hanwha Systems) ;
  • Moon, Young Shik (Department of computer science and engineering, Hanyang University)
  • Received : 2018.05.18
  • Accepted : 2018.06.12
  • Published : 2018.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

A single-image super-resolution is a process of restoring a high-resolution image from a low-resolution image. Recently, the super-resolution using the deep neural network has shown good results. In this paper, we propose a neural network structure that improves speed and performance over conventional neural network based super-resolution methods. To do this, we analyze the conventional neural network based super-resolution methods and propose solutions. The proposed method reduce the 5 stages of the conventional method to 3 stages. Then we have studied the optimal width and depth by experimenting on the width and depth of the network. Experimental results have shown that the proposed method improves the disadvantages of the conventional methods. The proposed neural network structure showed superior performance and speed than the conventional method.

단일 영상 초해상도는 하나의 저해상도 영상에서 고해상도 영상을 복원하는 과정이다. 최근 심층신경망을 적용한 초해상도 기법이 좋은 성과를 나타내고 있다. 본 논문에서는 기존의 심층신경망 기반 초해상도 복원 기법보다 속도와 성능을 개선한 신경망 구조를 제안한다. 이를 위해 기존 기법의 단점을 분석하고 해결책을 제시한다. 제안하는 방법은 기존 기법의 5단계를 3단계로 줄여 효율성을 높였으며, 네트워크의 폭과 깊이에 대한 실험을 통해 가장 효율적인 신경망 구조를 연구하였다. 제안하는 방법의 성능과 속도를 알아보기 위해 비교 실험을 진행하였다. 제안하는 방법은 $1024{\times}1024$ 영상을 초당 148장 복원하는 속도를 나타냈으며, 4가지 데이터에 대해 기존 방법보다 복원 성능이 우수하였다.

Keywords

References

  1. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image super-resolution as sparse representation of raw image patches," in IEEE Conference on Computer Vision and Pattern Recognition, June 2008. https://doi.org/10.1109/CVPR.2008.4587647
  2. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Transactions on Image Processing, Vol. 19, No. 11, pp.2861-2873, 2010. https://doi.org/10.1109/TIP.2010.2050625
  3. R. Timofte, V. D. Smet, and L. V. Gool, "A+: Adjusted anchored neighborhood regression for fast super-resolution," in Asian Conference on Computer Vision, Nov. 2014. https://doi.org/10.1007/978-3-319-16817-3_8
  4. C. Y. Yang and M. H. Yang, "Fast direct superresolution by simple functions," in IEEE Conference on Computer Vision and Pattern Recognition, Dec. 2013. https://doi.org/10.1109/ICCV.2013.75
  5. S. Schulter, C. Leistner, and H. Bischof, "Fast and accurate image upscaling with super-resolution forests," in IEEE Conference on Computer Vision and Pattern Recognition, June 2015. https://doi.org/10.1109/CVPR.2015.7299003
  6. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, No. 2, pp295-307, 2015. https://doi.org/10.1109/TPAMI.2015.2439281
  7. C. Dong, C. C. Loy, and X. Tang, "Accelerating the Super-Resolution Convolutional Neural Network," in European Conference on Computer Vision, pp.391-407, 2016. https://doi.org/10.1007/978-3-319-46475-6_25
  8. G. Freedman and R. Fattal, "Image and video upscaling from local self-examples," ACM Transactions on Graphics, Vol. 30, No. 2, 2011. https://doi.org/10.1063/1.4993002
  9. D. Glasner, S. Bagon, and M. Irani, "Super-resolution from a single image," in IEEE International Conference on Computer Vision, Sept. 2009. https://doi.org/10.1145/1944846.1944852
  10. A. Singh and N. Ahuja, "Super-resolution using subband self-similarity," in Asian Conference on Computer Vision, Nov. 2014. https://doi.org/10.1007/978-3-319-16808-1_37
  11. J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in IEEE Conference on Computer Vision and Pattern Recognition, June 2015. https://doi.org/10.1109/CVPR.2015.7299156
  12. J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in IEEE Conference on Computer Vision and Pattern Recognition, June 2016. https://doi.org/10.1109/CVPR.2016.182
  13. J. Kim, J. K. Lee, and K. M. Lee, "Deeply-Recursive Convolutional Network for Image Super-Resolution," in IEEE Conference on Computer Vision and Pattern Recognition, June 2016. https://doi.org/10.1109/CVPR.2016.181
  14. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, "Photo-realistic single image super-resolution using a generative adversarial network," in IEEE Conference on Computer Vision and Pattern Recognition, July 2017. https://doi.org/10.1109/CVPR.2017.19
  15. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in IEEE Conference on Computer Vision and Pattern Recognition, 2016. https://doi.org/10.1109/CVPR.2016.90
  16. K. He, X. Zhang, S. Ren, and J. Sun, "Identity Mappings in Deep Residual Networks," in European Conference on Computer Vision, 2016. https://doi.org/10.1007/978-3-319-46493-0_38
  17. S. Zagoruyko and N. Komodakis, "Wide Residual Networks," in Proceedings of the British Machine Vision Conference, pp.87.1-87.12. Sept.2016. https://dx.doi.org/10.5244/C.30.87
  18. Z. Wojna, V. Ferrari, S. Guadarrama, N. Silberman, L.-C. Chen, A. Fathi, and J. Uijlings, "The Devil is in the Decoder," arXiv preprint, arXiv:1707.05847, 2017. https://arxiv.org/abs/1707.05847
  19. M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Low-complexity single-image superresolution based on nonnegative neighbor embedding," in Proceedings of the 23rd British Machine Vision Conference, Sept. 2012. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.297.1474
  20. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in International Conference on Curves and Surfaces. June 2010. https://doi.org/10.1007/978-3-642-27413-8_47
  21. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 5, pp.898-916, 2011. https://doi.org/10.1109/TPAMI.2010.161
  22. E. Agustsson and R. Timofte "NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study," in IEEE Conference on Computer Vision and Pattern Recognition Workshops, July, 2017. http://doi.org/10.1109/CVPRW.2017.150
  23. Diederik P. Kingma, Jimmy Ba, "Adam: A Method for Stochastic Optimization," arXiv preprint, arXiv:1412. 6980, 2014. https://arxiv.org/abs/1412.6980