방송공학회논문지 (Journal of Broadcast Engineering)

  • 제27권3호
  • /
  • Pages.378-390
  • /
  • 2022
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

정보 증류 및 재귀적인 방식을 이용한 심층 학습법 기반 경량화된 초해상도 네트워크

Lightweight Super-Resolution Network Based on Deep Learning using Information Distillation and Recursive Methods

  • 우희조 (한국공학대학교 전자공학부) ;
  • 심지우 (한국공학대학교 전자공학부) ;
  • 김응태 (한국공학대학교 전자공학부)
  • Woo, Hee-Jo (Department of Electronics Engineering, Tech University of KOREA) ;
  • Sim, Ji-Woo (Department of Electronics Engineering, Tech University of KOREA) ;
  • Kim, Eung-Tae (Department of Electronics Engineering, Tech University of KOREA)
  • 투고 : 2022.04.07
  • 심사 : 2022.05.10
  • 발행 : 2022.05.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

최근 심층 합성 곱 신경망 학습의 발전에 따라 단일 영상 초해상도에 적용되는 심층 학습 기법들을 좋은 성과를 보여주고 있으며 깊은 네트워크의 강한 표현 능력으로 저해상도 영상과 고해상도 영상 사이의 복잡한 비선형 매핑이 가능해졌다. 하지만 과도한 합성곱 신경망의 사용으로 인해 증가하는 파라미터와 연산량으로 실시간 또는 저전력 장치에 적용하는데 제한이 있다. 본 논문은 정보 증류 방식을 이용하여 계층적인 특징을 조금씩 추출해내는 블록을 재귀적인 방식으로 사용하며 고주파수 잔여 정제 블록을 통해 더 정확한 고주파수 성분을 만들어 성능을 향상시키는 경량화된 네트워크인 Recursive Distillation Super Resolution Network (RDSRN) 를 제안한다. 제안하는 네트워크는 RDN과 비교했을 때 비슷한 화질의 영상을 복원하며 약 32배 적은 파라미터와 약 10배 적은 연산량을 가지고 약 3.5배 더 빠르게 영상을 복원하며 기존 경량화 네트워크 CARN과 비교했을 때 약 2.2배 적은 파라미터와 약 1.8배 빠른 처리시간으로 평균 0.16dB 더 좋은 성능을 만들어 냄을 확인 하였다.

With the recent development of deep composite multiplication neural network learning, deep learning techniques applied to single-image super-resolution have shown good results, and the strong expression ability of deep networks has enabled complex nonlinear mapping between low-resolution and high-resolution images. However, there are limitations in applying it to real-time or low-power devices with increasing parameters and computational amounts due to excessive use of composite multiplication neural networks. This paper uses blocks that extract hierarchical characteristics little by little using information distillation and suggests the Recursive Distillation Super Resolution Network (RDSRN), a lightweight network that improves performance by making more accurate high frequency components through high frequency residual purification blocks. It was confirmed that the proposed network restores images of similar quality compared to RDN, restores images 3.5 times faster with about 32 times fewer parameters and about 10 times less computation, and produces 0.16 dB better performance with about 2.2 times less parameters and 1.8 times faster processing time than the existing lightweight network CARN.

키워드

과제정보

This work was supported by the Technology development Program (S3025098) funded by the Ministry of SMEs and Startups(MSS, Korea) This work was partly supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea Government(MSIT) and Korea Institute for Advancement of Technology(KIAT) grant funded by the Korea Government(MOTIE) (P0008458, HRD Program for Industrial Innovation).

참고문헌

  1. X. Zhang and X. Wu, "Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation," in IEEE Transactions on Image Processing, pp. 887-896, 2008 doi: https://doi.org/10.1109/TIP.2008.924279
  2. Y. Romano, M. Protter and M. Elad, "Single Image Interpolation Via Adaptive Nonlocal Sparsity-Based Modeling," in IEEE Transactions on Image Processing, pp. 3085-3098, 2014 doi: https://doi.org/10.1109/TIP.2014.2325774
  3. K. Zhang, X. Gao, D. Tao and X. Li, "Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression," in IEEE Transactions on Image Processing, pp. 4544-4556, 2012 doi: https://doi.org/10.1109/TIP.2012.2208977
  4. V. Papyan and M. Elad, "Multi-Scale Patch-Based Image Restoration," in IEEE Transactions on Image Processing, pp. 249-261, 2016 doi: https://doi.org/10.1109/TIP.2015.2499698
  5. C. Ren, X. He and T. Q. Nguyen, "Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature," in IEEE Transactions on Image Processing, pp. 90-106, 2017 doi: https://doi.org/10.1109/TIP.2016.2619265
  6. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in CVPR, pp. 2472-2481, 2018. doi: https://doi.org/10.1109/CVPR.2018.00262
  7. Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," in CVPR, pp. 2790-2798, 2017. doi: https://doi.org/10.1109/CVPR.2017.298
  8. G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, "Densely Connected Convolutional Networks," in CVPR, pp. 2261-2269, 2017 doi: https://doi.org/10.1109/CVPR.2017.243
  9. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," In ICLR, 2015
  10. W. Shi, J. Caballero, F. Huszr, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in CVPR, pp. 1874-1883, 2016. doi: https://doi.org/10.1109/CVPR.2016.207
  11. C. Dong, C. C. Loy, K. He, and X. Tang. "Image Super-Resolution Using Deep Convolutional Networks," In Transactions on Pattern Analysis and Machine Intelligence, pp. 295-307, 2016.
  12. J. Kim, J. Kwon Lee, and K. Mu Lee, "Accurate image super resolution using very deep convolutional networks," in CVPR, pp. 1646-1654, 2016. doi: https://doi.org/10.1109/CVPR.2016.182
  13. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," ECCV, pp. 286-301, 2018. doi: https://doi.org/10.1007/978-3-030-01234-2_18
  14. N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and, lightweight super-resolution with cascading residual network," ECCV, 2018. doi: https://doi.org/10.1007/978-3-030-01249-6_16
  15. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in CVPR, 2016. doi: https://doi.org/10.1109/CVPR.2016.90
  16. Andrew G. Howard, Bo Chen, D. Kalenichenko, W. Wang, T. Weand, M. Andreetto, H. Adam, "MobileNets: Efficient Convolutional Neural Networks for mobile vision applications," in CVPR, 2017.
  17. Z. Hui, X. Wang, and X. Gao, "Fast and accurate single image super-resolution via information distillation network," IEEE Conference on Computer Vision and Pattern Recognition, pp. 723-731, 2018. doi: https://doi.org/10.1109/CVPR.2018.00082