Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Single Image Super-resolution using Recursive Residual Architecture Via Dense Skip Connections

고밀도 스킵 연결을 통한 재귀 잔차 구조를 이용한 단일 이미지 초해상도 기법

  • Chen, Jian (Department of Electronic Computer Engineering, Hanyang University) ;
  • Jeong, Jechang (Department of Electronic Computer Engineering, Hanyang University)
  • 진건 (한양대학교 전자통신컴퓨터공학부) ;
  • 정제창 (한양대학교 전자통신컴퓨터공학부)
  • Received : 2019.03.29
  • Accepted : 2019.06.26
  • Published : 2019.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, the convolution neural network (CNN) model at a single image super-resolution (SISR) have been very successful. The residual learning method can improve training stability and network performance in CNN. In this paper, we propose a SISR using recursive residual network architecture by introducing dense skip connections for learning nonlinear mapping from low-resolution input image to high-resolution target image. The proposed SISR method adopts a method of the recursive residual learning to mitigate the difficulty of the deep network training and remove unnecessary modules for easier to optimize in CNN layers because of the concise and compact recursive network via dense skip connection method. The proposed method not only alleviates the vanishing-gradient problem of a very deep network, but also get the outstanding performance with low complexity of neural network, which allows the neural network to perform training, thereby exhibiting improved performance of SISR method.

최근, 단일 이미지 초해상도 복원 기법(super-resolution)에서 컨볼루션 신경망 모델은 매우 성공적이다. 잔여 학습 기법은 컨볼루션 신경망 훈련의 안전성과 성능을 향상시킬 수 있다. 본 논문은 저해상도 입력 이미지에서 고해상도 목표 이미지로 비선형 매핑 학습을 위해 고밀도 스킵 연결(dense skip-connection)을 통한 재귀 잔차 구조를 이용한 단일 이미지 초해상도 복원 기법을 제안한다. 제안하는 단일 이미지 초해상도 복원 기법은 고밀도 스킵 연결 방식을 통해 재귀 잔차 학습 방법을 채택해서 깊은 신경망에서 학습이 어려운 문제를 완화하고 더 쉽게 최적화하기 위해 신경망 안에 불필요한 레이어를 제거한다. 제안하는 방법은 매우 깊은 신경망의 사라지는 변화도(vanishing gradient) 문제를 완화할 뿐만 아니고 낮은 복잡성으로 뛰어난 성능을 얻음으로써 단일 이미지 초해상도 복원 기법의 성능을 향상시킨다. 실험 결과를 통해 제안하는 알고리듬이 기존의 알고리듬 보다 결과가 더 우수함을 보인다.

Keywords

BSGHC3_2019_v24n4_633_f0001.png 이미지

그림 1. SRCNN 모델 구조 Fig. 1. SRCNN model structure

BSGHC3_2019_v24n4_633_f0002.png 이미지

그림 2. 잔차 학습의 구조 Fig. 2. The structure of residual learning

BSGHC3_2019_v24n4_633_f0003.png 이미지

그림 3. 재귀 잔차 블록의 구조. U는 재귀 블록 내 잔차 단위 개수이다. Fig. 3. Structures of our recursive residual block. U means number of residual units in the recursive block

BSGHC3_2019_v24n4_633_f0004.png 이미지

그림 4. 본 논문에서 제안된 CNN 모델의 구조 Fig. 4. The architecture of our proposed CNN model

BSGHC3_2019_v24n4_633_f0005.png 이미지

그림 5. Set5의 “butterfly” 영상의 2배 확대 실험 결과 Fig. 5. The result image of “butterfly” from Set5 with an upscaling factor=2

BSGHC3_2019_v24n4_633_f0006.png 이미지

그림 6. Set14의 “zebra” 영상의 2배 확대 실험 결과 Fig. 6. The result image of “zebra” from Set14 with an upscaling factor=2

표 1. Set5, Set14의 2배, 3배, 4배 확대 결과 평균 PSNR Table 1. Average PSNR for scale 2, 3, and 4 on datasets Set5, and Set14

BSGHC3_2019_v24n4_633_t0001.png 이미지

References

  1. W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, "Learning low-level vision," International Journal of Computer Vision, Vol.40, No.11, pp. 25-47. 2000. https://doi.org/10.1023/A:1026501619075
  2. D. Glasner, S. Bagon, and M. Irani, "Super-resolution from a single image," Proceeding of International Conference on Computer Vision, 2009.
  3. M. Irani, and S. Peleg, "Improving resolution by image registration," Graphical models and Image Processing, pp.231-239, 1991.
  4. S. Peled, and Y. L. Cun, "Super-resolution in MRI: application to human white matter fiber tract visualization by diffusion tensor imaging," Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine , Vol.45, No.1, pp. 29-35, 2001. https://doi.org/10.1002/1522-2594(200101)45:1<29::AID-MRM1005>3.0.CO;2-Z
  5. W. Shi, J. Caballero, C. Ledig, X. Zhuang, W. Bai, K. Bhatia, A. Marvao, T. Dawes, D. ORegan, and D. Rueckert, "Cardiac image super-resolution with global correspondence using multi-atlas patchmatch," Proceeding of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.9-16. 2013.
  6. W. Zou and P. C. Yuen, "Very low resolution face recognition problem," IEEE Transaction on Image Processing, Vol.21, No.1, pp.327-340, July 2011. https://doi.org/10.1109/TIP.2011.2162423
  7. B. K. Gunturk, A. U. Batur, Y. Altunbasak, M. H. Hayes, and R. M. Mersereau, "Eigenface-domain super-resolution for face recognition," IEEE Transaction on Image Processing, Vol.12, No.5, pp.597-606, 2003. https://doi.org/10.1109/TIP.2003.811513
  8. D. Y. Han, "Comparison of commonly used image interpolation methods," Proceeding of International Conference on Computer Science and Electronics Engineering, Mar 2013.
  9. C. E. Duchon, "Lanczos filtering in one and two dimensions," Journal of Applied Meteorology, pp.1016-1022, 1979.
  10. H. Chang, D. Y. Yeung, and Y. Xiong, "Super-resolution through neighbor embedding," Proceeding of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2004.
  11. C. G. Marco Bevilacqua, Aline Roumy, and M. L. A. Morel, "Low complecity single-image super-resolution based on nonnegative neighbor embedding," Proceeding of IEEE British Machine Vision Conference, 2012.
  12. R. Timofte, V. De Smet, and L. Van Gool, "A+: Adjusted anchored neighborhood regression for fast super-resolution," Proceeding of Asian Conference on Computer Vision (ACCV), Nov 2014.
  13. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Transaction on Image Processing, Vol.19, No.11, Nov 2010.
  14. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolution networks," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol.38, No.2, Feb 2016.
  15. C. Dong, C. C. Loy, K. He, and X. Tang, "Accelerating the super-resolution convolutional neural network." Proceeding of European Conference on Computer Vision (ECCV), 2016.
  16. J. Kim, J. Kwon, and K. Mu. Lee, "Accurate image super-resolution using very deep convolution networks," Proceeding of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  17. A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet calssification with deep convolutiuon neural network," Neural Information Processing Systems (NIPS), 2012.
  18. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. L. Cun, "Overfeat: Intergrated recognition, localization and detection using convolution networks," Proceeding of IEEE conference on Computer Vision and Pattern Recognition (CVPR), Dec 2013.
  19. K. He, X. Y. Zhang, S. Q. Ren, and J. S, "Deep residual learning for image recognition," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Dec 2015.
  20. G. H, Z. L, L. van der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," Proceeding of IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  21. S. Ioffe, and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," Procceding of International Conference on Machine Learning(ICML), pp 1097-1105, 2012.
  22. Nah, T. H. Kim, and K. M. Lee, "Deep multi-scale convolutional neural network for dynamic scene deblurring," Proceeding of IEEE conference on Computer Vision and Pattern Recognition (CVPR), Dec. 2016.
  23. R. Timofte, R. Rothe, and L. Cool, "Seven ways to improve example-based single image super resolution," Proceeding of IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp.1865-1873, Jun 2016.
  24. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," Proceeding of IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2001.
  25. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," Proceeding of International Conference on Curves and Surfaces. pp.711-730, 2012.