Browse > Article
http://dx.doi.org/10.3837/tiis.2022.09.003

Cascaded Residual Densely Connected Network for Image Super-Resolution  

Zou, Changjun (East China Jiaotong University)
Ye, Lintao (East China Jiaotong University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.9, 2022 , pp. 2882-2903 More about this Journal
Abstract
Image super-resolution (SR) processing is of great value in the fields of digital image processing, intelligent security, film and television production and so on. This paper proposed a densely connected deep learning network based on cascade architecture, which can be used to solve the problem of super-resolution in the field of image quality enhancement. We proposed a more efficient residual scaling dense block (RSDB) and the multi-channel cascade architecture to realize more efficient feature reuse. Also we proposed a hybrid loss function based on L1 error and L∞ error to achieve better L∞ error performance. The experimental results show that the overall performance of the network is effectively improved on cascade architecture and residual scaling. Compared with the residual dense net (RDN), the PSNR / SSIM of the new method is improved by 2.24% / 1.44% respectively, and the L∞ performance is improved by 3.64%. It shows that the cascade connection and residual scaling method can effectively realize feature reuse, improving the residual convergence speed and learning efficiency of our network. The L∞ performance is improved by 11.09% with only a minimal loses of 1.14% / 0.60% on PSNR / SSIM performance after adopting the new loss function. That is to say, the L∞ performance can be improved greatly on the new loss function with a minor loss of PSNR / SSIM performance, which is of great value in L∞ error sensitive tasks.
Keywords
Super-resolution; Deeping Learning; Residual scaling; Cascade architecture;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 Dun Y, Da Z, Yang S, et al., "Image Super-Resolution based on Residually Dense Distilled Attention Network," Neurocomputing, 443, 47-57, 2021.   DOI
2 Sun L, Liu Z, Sun X, et al., "Lightweight Image Super-Resolution via Weighted Multi-Scale Residual Network," IEEE/CAA Journal of Automatica Sinica, 8(7), 1271-1280, 2021.   DOI
3 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P., "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on image processing (TIP), 13(4), 600-612, 2004.   DOI
4 Lim B, Son S, Kim H, et al., "Enhanced Deep Residual Networks for Single Image SuperResolution," in Proc. of the Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 1134-1140, 2017.
5 Li J, Fang F, Li J, et al., "MDCN: Multi-Scale Dense Cross Network for Image Super-Resolution," IEEE Transactions on Circuits and Systems for Video Technology, 31(7), 2547-2561, 2021.   DOI
6 Lv X, Wang C, Fan X, et al., "A novel image super-resolution algorithm based on multi-scale dense recursive fusion network," Neurocomputing, 489, 98-111, 2022.   DOI
7 Hai H, Li P, Zou N, et al., "End-to-End Super-Resolution for Remote-Sensing Images Using an Improved Multi-Scale Residual Network," Remote Sensing, 13(4), 666, 2021.
8 Chu X, Chen L, Yu W., "NAFSSR: Stereo Image Super-Resolution Using NAFNet," Computer Science - Computer Vision and Pattern Recognition, 2022.
9 Wu H, Gui J, Zhang J, et al., "Pyramidal Dense Attention Networks for Lightweight Image SuperResolution," in Proc. of the Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
10 Wang M J, Yang X, Anisetti M, et al., "Image super-resolution via enhanced multi-scale residual network," Journal of Parallel and Distributed Computing, 152, 57-66, 2021.   DOI
11 Wang L, Du J, Gholipour A, et al., "3D dense convolutional neural network for fast and accurate single MR image super-resolution," Computerized Medical Imaging and Graphics, 93, 101973, 2021.
12 Gupta H, Mitra K., "Toward Unaligned Guided Thermal Super-Resolution," IEEE Transactions on Image Processing, 31, 2022.
13 Yang X, Zhang M, Li W, et al., "Visible-Assisted Infrared Image Super-Resolution Based on Spatial Attention Residual Network," IEEE Geoscience and Remote Sensing Letters, 19, 2021.
14 Vyas N, Kunne S, Fish T M, et al., "Protocol for image registration of correlative soft X-ray tomography and super-resolution structured illumination microscopy images," STAR Protocols, 2(2), 100529, 2021.
15 Park J H, Lee J, Lee K, et al., "FBRNN: Feedback Recurrent Neural Network for Extreme Image Super-Resolution," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2020.
16 Zha L, Yang Y, Lai Z, et al., "A Lightweight Dense Connected Approach with Attention on Single Image Super-Resolution," Electronics, 10(11), 1234, 2021.
17 Ahn N, Kang B, Sohn K A., "Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network," in Proc. of 15th European Conference, Conference on European Conference on Computer Vision, Springer, Munich, Germany, 2018.
18 Gopalakrishnan S, Ovireddy S., "Hybridisation of single-image super-resolution with edge-aware multi-focus image fusion for edge enrichment," IET Image Processing, 14(16), 2020.
19 Chao Dong, Chen Change Loy, Xiaoou Tang, "Accelerating the Super-Resolution Convolutional Neural Network," in Proc. of European Conference on Computer Vision (ECCV), 391-407, 2016.
20 Dong C, Loy C C, He K, et al., "Image super-resolution using deep convolutional networks," IEEE transactions on pattern analysis and machine intelligence, 38(2), 295-307, 2015.   DOI
21 Kim J, Lee J K, Lee K M, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks," in Proc. of the Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
22 Lai W S, Huang J B, Ahuja N, et al., "Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution," in Proc. of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5835-5843, 2017.
23 Tong T, Li G, Liu X, et al., "Image Super-Resolution Using Dense Skip Connections," in Proc. of the Conference on Computer Vision (ICCV 2017), 2017.
24 Huang G, Liu Z, Laurens V, et al., "Densely Connected Convolutional Networks," in Proc. of the Conference on Computer Vision and Pattern Recognition(CVPR), 2261-2269, 2017.
25 Zhang Y, Tian Y, Kong Y, et al., "Residual Dense Network for Image Super-Resolution," in Proc. of the Conference on Computer Vision and Pattern Recognition(CVPR), 2018.
26 Guo D, Xia Y, Xu L, et al., "Remote Sensing Image Super-resolution Using Cascade Generative Adversarial Nets," Neurocomputing, 443, 117-130, 2021.   DOI
27 Liu D, Li J, Yuan Q, "A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution," IEEE Transactions on Geoscience and Remote Sensing, 59(9), 7711-7725, 2021.   DOI
28 Zhu D, Qiu D, "Residual Dense Network for Medical Magnetic Resonance Images SuperResolution," Computer Methods and Programs in Biomedicine, 209, 2021.
29 Li Y, Cao J, Li Z, et al., "Lightweight Single Image Super-resolution with Dense Connection Distillation Network," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 17(1s), 1-17, 2021.
30 Yu H A, Ft A, Jin J, et al., "Dense channel splitting network for MR image super-resolution," Magnetic Resonance Imaging, 88, 53-61, 2022.   DOI
31 Li G, Zhu Y, "A Novel Dual Dense Connection Network for Video Super-resolution," Computer Science - Computer Vision and Pattern Recognition, 2022.
32 Zhang LX, Dong RM, Yuan S et al., "Making Low-Resolution Satellite Images Reborn: A Deep Learning Approach for Super-Resolution Building Extraction," Remote Sensing, 13(15), 2872, 2021.
33 Ma HU, Gong BC, Yu YZ, "Structure-aware Meta-fusion for Image Super-resolution," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 18(2), 1-25, 2022.
34 Fang N, Zhan Z., "High-resolution optical flow and frame-recurrent network for video superresolution and deblurring," Neurocomputing, 489, 128-138, 2022.   DOI
35 Zhou Y, Zhang Y, Xie X, et al., "Image super-resolution based on dense convolutional autoencoder blocks," Neurocomputing, 423, 98-109, 2021.   DOI
36 Lv Y, Ma H, Li J, et al., "Fusing dense and ReZero residual networks for super-resolution of retinal images," Pattern Recognition Letters, 149, 120-129, 2021.   DOI
37 Chen H, He X, Qing L, et al., "Real-world single image super-resolution: A brief review," Information Fusion, 79(3), 124-145, 2022.   DOI
38 Shah Z H, M Muller, Wang T C, et al., "Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images," Photonics Research, 9(5), B168-B181, 2021.   DOI
39 Jiang M, Zhi M, Yang L I, et al., "Super-resolution reconstruction of MR image with self-attention based generate adversarial network algorithm," Scientia Sinica Informationis, 51(6), 959-970, 2021.   DOI
40 Hou M, He X, Dou F, et al., "Semi-supervised image super-resolution with attention CycleGAN," IET Image Processing, 16(4), 1181-1193, 2022.   DOI
41 D Qiu, L Zheng, J Zhu, et al., "Multiple improved residual networks for medical image superresolution," Future Generation Computer Systems, 116, 200-208, 2021.   DOI
42 D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. of the Conference on Computer Vision (ICCV), 2021.
43 Qiu D, Zheng L, Zhu J, et al., "Multiple improved residual networks for medical image superresolution," Future Generation Computer Systems, 116, 200-208, 2021.   DOI
44 B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image super-resolution," in Proc. of the Conference on Computer Vision and Pattern Recognition Workshop (CVPRW), 1- 7, 2017.
45 He K, Zhang X, Ren S, et al., "Deep Residual Learning for Image Recognition," in Proc. of the Conference on Computer Vision and Pattern Reco-gnition (CVPR), 1-12, 2015.
46 Yang G, Wang Y, Yi C, et al., "A New super-resolution restoration method with Generated Adversarial Network for underground video images in coal mines," Journal of Physics: Conference Series, 2031(1), 012011, 2021.
47 Ledig C, Theis L, F Huszar, et al., "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," in Proc. of the Conference on Computer Vision and Pattern Recognition(CVPR), 7, 105-114, 2017.
48 Hu X, Mu H, Zhang X, et al., "Meta-SR: A Magnification-Arbitrary Network for SuperResolution," in Proc. of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
49 M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding," in Proc. of Conference on British Machine Vision Conference(BMVC), 135-1-135-10, 2012.
50 R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Proc. of 7th International Conference Curves Surface, 711-730, 2010.
51 Qiu C, Yao Y, Du Y, "Nested Dense Attention Network for Single Image Super-Resolutionn" in Proc. of the 2021 International Conference on Multimedia Retrieval, 250-258, 2021.
52 J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed selfexemplars," in Proc. of the Conference on Computer Vision and Pattern Recognition (CVPR), 5197-5206, 2015.
53 Cai J, Gu S, Timofte R, et al., "NTIRE 2019 Challenge on Real Image Super-Resolution: Methods and Results," in Proc. of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019.
54 Huynh-Thu Q, Ghanbari M, "Scope of validity of PSNR in image/video quality assessment," Electronics Letters, 44(13), 800-801, 2008.   DOI
55 SAHEBI. Super resolution Dataset [EB/OL]. [Online]. Available: https://www.kaggle.com/datasets/msahebi/super-resolution. 2022-4-19
56 Zhao D, Zhang F, Wang W, et al., "Medical Images Super Resolution Reconstruction based on Residual Network," in Proc. of ICCAI 2021: 2021 7th International Conference on Computing and Artificial IntelligenceApril, 119-126, 2021.
57 Wang M J, Yang X, Anisetti M, et al., "Image super-resolution via enhanced multi-scale residual network," Journal of Parallel and Distributed Computing, 152, 57-66, 2021.   DOI