Browse > Article
http://dx.doi.org/10.3837/tiis.2021.05.013

Single Low-Light Ghost-Free Image Enhancement via Deep Retinex Model  

Liu, Yan (School of Computer and Communication Engineering, Zhengzhou University of Light Industry)
Lv, Bingxue (School of Computer and Communication Engineering, Zhengzhou University of Light Industry)
Wang, Jingwen (School of Computer and Communication Engineering, Zhengzhou University of Light Industry)
Huang, Wei (School of Computer and Communication Engineering, Zhengzhou University of Light Industry)
Qiu, Tiantian (School of Computer and Communication Engineering, Zhengzhou University of Light Industry)
Chen, Yunzhong (School of Electronic Information and Electrical Engineering, Shanghai Jiaotong University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.5, 2021 , pp. 1814-1828 More about this Journal
Abstract
Low-light image enhancement is a key technique to overcome the quality degradation of photos taken under scotopic vision illumination conditions. The degradation includes low brightness, low contrast, and outstanding noise, which would seriously affect the vision of the human eye recognition ability and subsequent image processing. In this paper, we propose an approach based on deep learning and Retinex theory to enhance the low-light image, which includes image decomposition, illumination prediction, image reconstruction, and image optimization. The first three parts can reconstruct the enhanced image that suffers from low-resolution. To reduce the noise of the enhanced image and improve the image quality, a super-resolution algorithm based on the Laplacian pyramid network is introduced to optimize the image. The Laplacian pyramid network can improve the resolution of the enhanced image through multiple feature extraction and deconvolution operations. Furthermore, a combination loss function is explored in the network training stage to improve the efficiency of the algorithm. Extensive experiments and comprehensive evaluations demonstrate the strength of the proposed method, the result is closer to the real-world scene in lightness, color, and details. Besides, experiments also demonstrate that the proposed method with the single low-light image can achieve the same effect as multi-exposure image fusion algorithm and no ghost is introduced.
Keywords
image enhancement; low-light image; Retinex model; image decomposition; super-resolution;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Jobson, D.J. and Z. Rahman, "A multiscale retinex for bridging the gap between color images and the human observation of scenes," IEEE Transactions on Image Processing, vol. 6(7), pp. 965- 976, 1997.   DOI
2 Wang, L.W., et al., "Lightening Network for Low-Light Image Enhancement," IEEE Transactions on Image Processing, vol. 29, pp. 7984-7996, 2020.   DOI
3 Yang, J., et al., "Enhancement of Low Light Level Images with coupled dictionary learning," in Proc. of International Conference on Pattern Recognition, 2017.
4 Ren, W., et al., "Low-Light Image Enhancement via a Deep Hybrid Network," IEEE Transactions on Image Processing, vol. 28(9), pp. 4364-4375, 2019.   DOI
5 Land, E.H.E., "Lightness and Retinex Theory," Journal of the Optical Society of America, vol. 61(1), pp. 1-11, 1971.   DOI
6 Lai, W.S., et al., "Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 41(11), pp. 2599-2613, 2019.   DOI
7 Lv, F., et al., "MBLLEN: Low-Light Image/Video Enhancement Using CNNs," in Proc. of BMVC, 2018.
8 Venkatanath, N., et al., "Blind image quality evaluation using perception based features," in Proc. of 2015 Twenty First National Conference on Communications (NCC), IEEE, 2015.
9 Ying, Z., et al., "A New Low-Light Image Enhancement Algorithm Using Camera Response Model," in Proc. of 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017.
10 Li, J., et al., "Luminance-aware Pyramid Network for Low-light Image Enhancement," IEEE Transactions on Multimedia, 2020.
11 Wenhan, Y., et al., "Reference Guided Deep Super-Resolution via Manifold Localized External Compensation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 29(5), pp. 1270-1283, 2019.   DOI
12 Fu, L., et al., "Learning Robust Discriminant Subspace Based on Joint L2, p-and L2, s-Norm Distance Metrics," IEEE Transactions on Neural Networks and Learning Systems, 2020.
13 Lore, K.G., A. Akintayo, and S. Sarkar, "LLNet: A Deep Autoencoder Approach to Natural Lowlight Image Enhancement," Pattern Recognition, vol. 61, pp. 650-662, 2017.   DOI
14 He, A., et al., "A twofold siamese network for real-time object tracking," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
15 Fan, D.-P., et al., "Concealed object detection," arXiv preprint arXiv:2102.10274, 2021.
16 Boulila, W., et al., "RS-DCNN: A novel distributed convolutional-neural-networks basedapproach for big remote-sensing image classification," Computers and Electronics in Agriculture, vol. 182, pp. 106014, 2021.   DOI
17 Rahman, Z., D.J. Jobson, and G.A. Woodell., "Multi-scale retinex for color image enhancement," in Proc. of 3rd IEEE International Conference on Image Processing, 2002.
18 Xu, Y., et al., "A novel multi-scale fusion framework for detail-preserving low-light image enhancement," Information Sciences, vol. 548, pp. 378-397, 2021.   DOI
19 Lin, J., et al., "Fast multi-view image rendering method based on reverse search for matching," Optik, vol. 180, pp. 953-961, 2019.   DOI
20 Yang, et al., "Video super-resolution based on spatial-temporal recurrent residual networks," Computer Vision & Image Understanding, vol. 168, pp. 79-92, 2018.   DOI
21 Dong, X., Y.A. Pang, and J.G. Wen., "Fast efficient algorithm for enhancement of low lighting video," in Proc. of IEEE International Conference on Multimedia & Expo, 2011.
22 Tan, R.T., "Attentive Generative Adversarial Network for Raindrop Removal from A Single Image," in Proc. of CVPR 2018, 2018.
23 He, C., et al., "TSLRLN: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising," Signal Processing, vol. 184, pp. 108060, 2021.   DOI
24 Shen, L., et al., "Msr-net: Low-light image enhancement using deep convolutional network," arXiv preprint arXiv:1711.02488, 2017.
25 Chen, C., et al., "Learning to See in the Dark," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
26 Lai, W.-S., et al., "Fast and accurate image super-resolution with deep laplacian pyramid networks," IEEE transactions on pattern analysis and machine intelligence, vol. 41(11), pp. 2599-2613, 2019.   DOI
27 Guo, X., Y. Li, and H. Ling, "LIME: Low-Light Image Enhancement via Illumination Map Estimation," IEEE Trans Image Process, vol. 26(2), pp. 982-993, 2017.   DOI
28 Wei, C., et al., "Deep retinex decomposition for low-light enhancement," arXiv preprint arXiv:1808.04560, 2018.
29 Zhang, Y., J. Zhang, and X. Guo, "Kindling the Darkness: A Practical Low-light Image Enhancer," in Proc. of the 27th ACM International Conference on Multimedia, pp. 1632-1640, 2019.
30 Mittal, A., et al., "Making a "Completely Blind" Image Quality Analyzer," IEEE Signal Processing Letters, vol. 20(3), pp. 209-212, 2013.   DOI
31 Guo, L., et al., "Learned snakes for 3D image segmentation," Signal Processing, vol. 183, p. 108013, 2021.   DOI
32 Wang, P., E. Fan, and P. Wang, "Comparative analysis of image classification algorithms based on traditional machine learning and deep learning," Pattern Recognition Letters, vol. 141, pp. 61-67, 2021.   DOI
33 Cai, J., S. Gu, and L. Zhang, "Learning a Deep Single Image Contrast Enhancer from MultiExposure Images," IEEE Transactions on Image Processing, vol. 27(4), pp. 2049-2062, 2018.   DOI
34 Li, L., et al., "A low-light image enhancement method for both denoising and contrast enlarging," in Proc. of IEEE International Conference on Image Processing, 2015.
35 Wang, S., et al., "Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images," IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, vol. 22(9), pp. 3538 - 3548, 2013.   DOI
36 Alghamdi, M., et al., "Transfer Deep Learning for Reconfigurable Snapshot HDR Imaging Using Coded Masks," in Porc. of Computer Graphics Forum, Wiley Online Library, 2021.
37 Song, X., et al., "Multi-scale joint network based on Retinex theory for low-light enhancement," Signal, Image and Video Processing, pp. 1-8, 2021.
38 Pei, B., Y. Peng, and Y. Luo, "A method of detecting defects of smart meter LCD screen based on LSD and deep learning," Multimedia Tools and Applications, pp. 1-18, 2021.
39 Liu, J., et al., "Erase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
40 Guo, C., et al., "Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
41 Choi, J., et al., "Context-aware deep feature compression for high-speed visual tracking," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
42 Shah, K.I., et al., "Autonomous Parking-Lots Detection with Multi-Sensor Data Fusion Using Machine Deep Learning Techniques," Cmc -Tech Science Press-, vol. 66(2), pp. 1595-1612, 2021.
43 Niu, Y., et al., "HDR-GAN: HDR image reconstruction from multi-exposed ldr images with large motions," IEEE Transactions on Image Processing, vol. 30, pp. 3885-3896, 2021.   DOI
44 Celik, T. and T. Tjahjadi, "Contextual and Variational Contrast Enhancement," IEEE Transactions on Image Processing, vol. 20(12), pp. 3431-3441, 2011.   DOI
45 Yang, W., et al., "From Fidelity to Perceptual Quality: A Semi-Supervised Approach for LowLight Image Enhancement," in Proc. of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3063-3072, 2020.
46 Kousik, N., et al., "Improved salient object detection using hybrid Convolution Recurrent Neural Network," Expert Systems with Applications, vol. 166, pp. 114064, 2021.   DOI
47 Zoran, D., et al., "Towards robust image classification using sequential attention models," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9483-8482, 2020.
48 Land, Edwin, and H., "The Retinex Theory of Color Vision," Scientific American, 1977.
49 Jobson, D.J. and Z. Rahman, "Properties and performance of a center/surround retinex," IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, vol. 6(3), pp. 451-462, 1997.   DOI