Browse > Article
http://dx.doi.org/10.5909/JBE.2020.25.6.898

A Study on Various Attention for Improving Performance in Single Image Super Resolution  

Mun, Hwanbok (Department of Computer Science, Kookmin University)
Yoon, Sang Min (College of Computer Science, Kookmin University)
Publication Information
Journal of Broadcast Engineering / v.25, no.6, 2020 , pp. 898-910 More about this Journal
Abstract
Single image-based super-resolution has been studied for a long time in computer vision because of various applications. Various deep learning-based super-resolution algorithms are introduced recently to improve the performance by reducing side effects like blurring and staircase effects. Most deep learning-based approaches have focused on how to implement the network architecture, loss function, and training strategy to improve performance. Meanwhile, Several approaches using Attention Module, which emphasizes the extracted features, are introduced to enhance the performance of the network without any additional layer. Attention module emphasizes or scales the feature map for the purpose of the network from various perspectives. In this paper, we propose the various channel attention and spatial attention in single image-based super-resolution and analyze the results and performance according to the architecture of the attention module. Also, we explore that designing multi-attention module to emphasize features efficiently from various perspectives.
Keywords
Super-Resolution; Attention Module; Image Decomposition;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. Dong, C. Loy, K. He, "Image Super-Resolution Using Deep Convolutional Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.38, No.2 pp.295-307, Feb 2016.   DOI
2 J. Hu, L. Shen, G. Sun, "Squeeze-and-excitation networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, USA, pp.7132-7141, 2018.
3 L. Chen, H. Zhang, J. Xiao, L. Nie, J. Shao, W. Liu, T. Chua, "Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, pp.6298-6306, 2017.
4 Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, Y. Fu, "Image super-resolution using very deep residual channel attention networks," Proceedings of the European Conference on Computer Vision, Munich, Germany, pp.286-301, 2018.
5 T. Dai, J. Cai, Y. Zhang, S. Xia, L. Zhang, "Second-order attention network for single image super-resolution," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 11065-11074, 2019.
6 J. Kim, J. Kwon, K. Lee, "Accurate image super-resolution using very deep convolutional networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp.1646-1654, 2016.
7 J. Kim, J. Kwon Lee, K. Lee "Deeply-recursive convolutional network for image super-resolution," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp.1637-1645, 2016.
8 Y. Tai, J. Yang, X. Liu, "Image super-resolution via deep recursive residual network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, pp. 3147-3155, 2017.
9 W. Shi, J. Caballero, F. Huszar, J. Totz, A. Aitken, R. Bishop, D. Rueckert, Z. Wang, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp.1874-1883, 2016.
10 B. Lim, S. Son, H. Kim, S. Nah, K. Lee, "Enhanced deep residual networks for single image super-resolution," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, Hawaii, USA, pp.1132-1140, 2017.
11 Y. Tai, J. Yang, X. Liu, C. Xu, "Memnet: A persistent memory network for image restoration," Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 4539-4547, 2017.
12 E. Agustsson, R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, Hawaii, USA, pp.1100-1121, 2017.
13 T. Tong, G. Li, X. Liu, Q. Gao, "Image super-resolution using dense skip connections," Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 4799-4807, 2017.
14 Y. Hu, J. Li, Y. Huang, X. Gao, "Channel-wise and spatial feature modulation network for single image super-resolution," IEEE Transactions on Circuits and Systems for Video Technology, Vol.30, No.11, pp.3911-3927, Nov 2020.   DOI
15 Z. Chen, V. Badrinarayanan, C. Lee, A. Rabinovich, "Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks," Proceedings of the 35th International Conference on Machine Learning, PMLR, Vol.80, pp.794-803, 2018.
16 J. Song, H. Cho, J. Yoon, S. Yoon, "Structure adaptive total variation minimization-based image decomposition," IEEE Transactions on Circuits and Systems for Video Technology, Vol.28, No.9, pp.2164-2176, Sep 2018.   DOI