Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Single Image Super Resolution Based on Residual Dense Channel Attention Block-RecursiveSRNet

잔여 밀집 및 채널 집중 기법을 갖는 재귀적 경량 네트워크 기반의 단일 이미지 초해상도 기법

  • Woo, Hee-Jo (Department of Electronics Engineering, Korea Polytechnic University) ;
  • Sim, Ji-Woo (Department of Electronics Engineering, Korea Polytechnic University) ;
  • Kim, Eung-Tae (Department of Electronics Engineering, Korea Polytechnic University)
  • 우희조 (한국산업기술대학교 전자공학부) ;
  • 심지우 (한국산업기술대학교 전자공학부) ;
  • 김응태 (한국산업기술대학교 전자공학부)
  • Received : 2021.05.06
  • Accepted : 2021.06.14
  • Published : 2021.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

With the recent development of deep convolutional neural network learning, deep learning techniques applied to single image super-resolution are showing good results. One of the existing deep learning-based super-resolution techniques is RDN(Residual Dense Network), in which the initial feature information is transmitted to the last layer using residual dense blocks, and subsequent layers are restored using input information of previous layers. However, if all hierarchical features are connected and learned and a large number of residual dense blocks are stacked, despite good performance, a large number of parameters and huge computational load are needed, so it takes a lot of time to learn a network and a slow processing speed, and it is not applicable to a mobile system. In this paper, we use the residual dense structure, which is a continuous memory structure that reuses previous information, and the residual dense channel attention block using the channel attention method that determines the importance according to the feature map of the image. We propose a method that can increase the depth to obtain a large receptive field and maintain a concise model at the same time. As a result of the experiment, the proposed network obtained PSNR as low as 0.205dB on average at 4× magnification compared to RDN, but about 1.8 times faster processing speed, about 10 times less number of parameters and about 1.74 times less computation.

최근 심층 합성 곱 신경망 학습의 발전에 따라 단일 이미지 초해상도에 적용되는 심층 학습 기법들은 좋은 성과를 보여주고 있다. 현존하는 딥러닝 기반 초해상도 기법들 중 하나로 잔여 밀집 블록을 이용하여 초기의 특징 정보를 마지막 계층에 전달하여 이후의 계층들이 이전의 계층들의 입력정보를 사용하여 복원하는 RDN(Residual Dense Network)이 있다. 하지만 계층적인 모든 특징을 연결하여 학습하고 다수의 잔여 밀집 블록을 쌓게 되면 좋은 성능에도 불구하고 많은 파라미터의 수와 연산량을 가지게 되어 느린 처리 속도와 네트워크를 학습하는데 많은 시간이 소요되고 모바일 시스템에 적용이 어렵다는 단점을 가지고 있다. 본 논문에서는 이전의 정보를 다시 사용하는 연속 메모리 구조인 잔여 밀집 구조와 이미지의 특징맵에 따라 중요도를 결정해주는 채널 집중 기법을 이용한 잔여밀집 채널 집중 블록을 재귀적인 방식으로 사용하여 추가적인 파라미터 없이 네트워크의 깊이를 늘려 큰 수용 영역을 얻으며 동시에 간결한 모델을 유지할 수 있는 방식을 제안한다. 실험 결과 제안하는 네트워크는 RDN과 비교 하였을 때 4배 확대 배율에서 평균적으로 PSNR 0.205dB만큼 낮지만 약 1.8배 더 빠른 처리속도, 약 10배 더 적은 파라미터의 수와 약 1.74배 더 적은 연산량을 갖는 것을 실험을 통해 확인하였다.

Keywords

Acknowledgement

This work was supported by the 2021 sabbatical year research grant of the Korea Polytechnic University. This work was supported by the Technology development Program (S3025098) funded by the Ministry of SMEs and Startups(MSS, Korea).

References

  1. X. Zhang and X. Wu, "Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation," in IEEE Transactions on Image Processing, pp. 887-896, 2008
  2. Y. Romano, M. Protter and M. Elad, "Single Image Interpolation Via Adaptive Nonlocal Sparsity-Based Modeling," in IEEE Transactions on Image Processing, pp. 3085-3098, 2014 https://doi.org/10.1109/TIP.2014.2325774
  3. W. Ye and K. Ma, "Convolutional Edge Diffusion for Fast Contrast-guided Image Interpolation," in IEEE Signal Processing Letters, pp. 1260-1264, 2016
  4. H. A. Aly and E. Dubois, "Image up-sampling using total-variation regularization with a new observation model," in IEEE Transactions on Image Processing, pp. 1647-1659, 2005
  5. K. Zhang, X. Gao, D. Tao and X. Li, "Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression," in IEEE Transactions on Image Processing, pp. 4544-4556, 2012
  6. V. Papyan and M. Elad, "Multi-Scale Patch-Based Image Restoration," in IEEE Transactions on Image Processing, pp. 249-261, 2016
  7. C. Ren, X. He and T. Q. Nguyen, "Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature," in IEEE Transactions on Image Processing, pp. 90-106, 2017
  8. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in CVPR, pp. 2472-2481, 2018.
  9. Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," in CVPR, pp. 2790-2798, 2017.
  10. G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, "Densely Connected Convolutional Networks," in CVPR, pp. 2261-2269, 2017
  11. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," In ICLR, 2015
  12. W. Shi, J. Caballero, F. Huszr, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in CVPR, pp. 1874-1883, 2016.
  13. Bevilacqua, A. Roumy, C. Guillemot and M.-L. A. Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," In Proceedings British Machine Vision Conference, pp. 135.1-135.10, 2012.
  14. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," In International conference on Curves and Surfaces, pp. 711-730, 2012.
  15. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," In International Conference on Computer Vision, pp. 416-423, 2001.
  16. J.-B. Huang, A. Singh, and N. Ahuja. "Single Image Super-Resolution From Transformed Self -Exemplars," In Conference on Computer Vision and Pattern Recognition, pp. 5197-5206, 2015.
  17. C. Dong, C. C. Loy, K. He, and X. Tang. "Image Super-Resolution Using Deep Convolutional Networks," In Transactions on Pattern Analysis and Machine Intelligence, pp. 295-307, 2016.
  18. J. Kim, J. Kwon Lee, and K. Mu Lee, "Accurate image super resolution using very deep convolutional networks," in CVPR, pp. 1646-1654, 2016.
  19. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," ECCV, pp. 286-301, 2018.