Journal of the Korea Convergence Society (한국융합학회논문지)

Korea Convergence Society (한국융합학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning-based Super Resolution Method Using Combination of Channel Attention and Spatial Attention

채널 강조와 공간 강조의 결합을 이용한 딥 러닝 기반의 초해상도 방법

  • Lee, Dong-Woo (Dept. of Plasma Bio Display, KwangWoon University) ;
  • Lee, Sang-Hun (Ingenium College of Liberal Arts, KwangWoon University) ;
  • Han, Hyun Ho (College of General Education, University of Ulsan)
  • 이동우 (광운대학교 플라즈마바이오디스플레이학부) ;
  • 이상훈 (광운대학교 인제니움학부) ;
  • 한현호 (울산대학교 교양대학)
  • Received : 2020.10.20
  • Accepted : 2020.12.20
  • Published : 2020.12.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we proposed a deep learning based super-resolution method that combines Channel Attention and Spatial Attention feature enhancement methods. It is important to restore high-frequency components, such as texture and features, that have large changes in surrounding pixels during super-resolution processing. We proposed a super-resolution method using feature enhancement that combines Channel Attention and Spatial Attention. The existing CNN (Convolutional Neural Network) based super-resolution method has difficulty in deep network learning and lacks emphasis on high frequency components, resulting in blurry contours and distortion. In order to solve the problem, we used an emphasis block that combines Channel Attention and Spatial Attention to which Skip Connection was applied, and a Residual Block. The emphasized feature map extracted by the method was extended through Sub-pixel Convolution to obtain the super resolution. As a result, about PSNR improved by 5%, SSIM improved by 3% compared with the conventional SRCNN, and by comparison with VDSR, about PSNR improved by 2% and SSIM improved by 1%.

본 논문은 채널 강조(Channel Attentin)와 공간 강조(Spatial Attention) 방법을 결합한 딥 러닝 기반의 초해상도 방법을 제안하였다. 초해상도 과정에서 질감, 특징과 같은 주변 픽셀의 변화량이 큰 고주파 성분의 복원이 중요하다. 채널 강조와 공간 강조를 결합한 특징 강조를 이용한 초해상도 방법을 제안하였다. 기존의 CNN(Convolutional Neural Network) 기반의 초해상도 방법은 깊은 네트워크의 학습이 어려우며, 고주파 성분의 강조가 부족하여 윤곽선이 흐려지거나 왜곡이 발생한다. 문제를 해결하기 위해 스킵-커넥션(Skip Connection)을 적용한 채널 강조와 공간 강조를 결합한 강조 블록과 잔차 블록(Residual Block)을 사용하였다. 방법으로 추출한 강조된 특징 맵을 부-픽셀 컨볼루션(Sub-pixel Convolution)을 통해 특징맵을 확장하여 초해상도를 진행하였다. 이를 통해 기존의 SRCNN과 비교하여 약 PSNR는 5%, SSIM은 3% 향상되었으며 VDSR과 비교를 통해 약 PSNR는 2%, SSIM은 1% 향상된 결과를 보였다.

Keywords

References

  1. M. K. Kwon & H. S. Yang. (2017). Performance Improvement of Object Recognition System in Broadcast Media Using Hierarchical CNN. Journal of Digital Convergence, 15(3), 201-209. https://doi.org/10.14400/JDC.2017.15.3.201
  2. B. H. Kim. (2016). Design of Image Tracking System Using Location Determination Technology. Journal of Digital Convergence, 14(11), 143-148. https://doi.org/10.14400/JDC.2016.14.11.143
  3. D. W. Lee, S. H. Lee, & H. H. Han. A Study on Super Resolution Method using Encoder-Decoder with Residual Learning. Journal of Advanced Research in Dynamical and Control Systems, 11(7), pp.2426-2433. November. 2019.
  4. D. W. Lee, S. H. Lee, H. H. Han & G. S. Chae. (2019). Improved Skin Color Extraction Based on Flood Fill for Face Detection. Journal of the Korea Convergence Society, 10(6), 7-14. https://doi.org/10.15207/JKCS.2019.10.6.007
  5. H. J. Kim, Y. S. Park, K. B. Kim & S. H. Lee. (2019). Modified HOG Feature Extraction for Pedestrian Tracking. Journal of the Korea Convergence Society, 10(3), 39-47. https://doi.org/10.15207/JKCS.2019.10.3.039
  6. K. He, X. Zhang, S. Ren & J. Sun.. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
  7. J. Hu, L. Shen & G. Sun. (2018). Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).
  8. S. Woo, J. Park, J. Y. Lee & I. S. Kweon. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).
  9. W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop & Z. Wang. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883).
  10. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva & A. Torralba. (2016). Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2921-2929).
  11. J. Park, S. Woo, J. Y. Lee & I. S. Kweon. (2018). Bam: Bottleneck attention module. arXiv preprint arXiv:1807.06514.
  12. R. Keys. (1981). Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing, 29(6), 1153-1160. https://doi.org/10.1109/TASSP.1981.1163711
  13. D. P. Kingma & J. Ba. (2014). Adam: a method for stochastic optimization, 1-15. arXiv preprint arXiv:1412.6980.
  14. C. Dong, C. C. Loy, K. He, & X. Tang. (2015). Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2), 295-307. https://doi.org/10.1109/TPAMI.2015.2439281
  15. J. Kim, J. Kwon Lee & K. Mu Lee. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654).