[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.3837/tiis.2022.02.010

Attention-based for Multiscale Fusion Underwater Image Enhancement

Huang, Zhixiong (College of Electronic and Communications Engineering, Shandong Technology and Business University)
Li, Jinjiang (Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Shandong Technology and Business University)
Hua, Zhen (College of Electronic and Communications Engineering, Shandong Technology and Business University)

Publication Information

KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.2, 2022 , pp. 544-564 More about this Journal

Abstract

Underwater images often suffer from color distortion, blurring and low contrast, which is caused by the propagation of light in the underwater environment being affected by the two processes: absorption and scattering. To cope with the poor quality of underwater images, this paper proposes a multiscale fusion underwater image enhancement method based on channel attention mechanism and local binary pattern (LBP). The network consists of three modules: feature aggregation, image reconstruction and LBP enhancement. The feature aggregation module aggregates feature information at different scales of the image, and the image reconstruction module restores the output features to high-quality underwater images. The network also introduces channel attention mechanism to make the network pay more attention to the channels containing important information. The detail information is protected by real-time superposition with feature information. Experimental results demonstrate that the method in this paper produces results with correct colors and complete details, and outperforms existing methods in quantitative metrics.

Keywords

Underwater image enhancement; Multiscale fusion; Convolutional neural network; Attention mechanism; Local binary pattern;

Citations & Related Records

Reference

1	Dai. C, Lin. M, Wu, X, et al, "Single underwater image restoration by decomposing curves of attenuating color," Optics & Laser Technology, vol. 123, no. 1, 2020, Article No. 105947.
2	Berman. D, Levy. D, Avidan. S, et al, "Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 8, pp. 2822-2837, 2021.
3	Silberman. N, Hoiem. D, Kohli. P, et al, "Indoor segmentation and support inference from rgbd images," in Proc. of European Conference on Computer Vision, Springer, Berlin, Heidelberg, pp. 746-760, 2012.
4	Panetta. K, Gao. C, Again. S, "Human-visual-system-inspired underwater image quality measures," IEEE Journal of Oceanic Engineering, vol. 41, no. 3, pp. 541-551, 2016. DOI
5	Zhang. W, Dong. L, Zhang. T, et al, "Enhancing underwater image via color correction and bi-interval contrast enhancement," Signal Processing: Image Communication, vol. 90, no. 1, 2021, Article No. 116030.
6	Hegde. D, Desai. C, Tabib. R, et al, "Adaptive cubic spline interpolation in CIELAB color space for underwater image enhancement," Procedia Computer Science, vol. 171, no. 1, pp. 52-61, 2020. DOI
7	Chen. C, Chen. Q, Xu. J, et al, "Learning to see in the dark," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3291-3300, 2018.
8	Ancuti. C. O, Ancuti. C, Vleeschouwer. C. D, et al, "Color balance and fusion for underwater image enhancement," IEEE Transactions on image processing, vol. 27, no. 1, pp. 379-393, 2018. DOI
9	Gao. S. B, Zhang. M, Zhao. Q, et al, "Underwater image enhancement using adaptive retinal mechanisms," IEEE Transactions on Image Processing, vol. 28, no. 11, pp. 5580-5595, 2019. DOI
10	Ancuti. C. O, Ancuti. C, De. Vleeschouwer. C, et al, "Color channel compensation (3C): A fundamental pre-processing step for image enhancement," IEEE Transactions on Image Processing, vol. 29, no. 1, pp. 2653-2665, 2019.
11	W. Song, Y. Wang, D. Huang, et al, "Enhancement of underwater images with statistical model of background light and optimization of transmission map," IEEE Transactions on Broadcasting, vol. 66, no. 1, pp. 153-169, 2020. DOI
12	Yang. M, Sowmya. A, "An underwater color image quality evaluation metric," IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 6062-6071, 2015. DOI
13	Liang. Z, Wang. Y, Ding. X, et al, "Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing," Neurocomputing, vol. 425, no. 1, pp. 160-172, 2021. DOI
14	Kar. A, Dhara. S. K, Sen. D, et al, "Zero-Shot Single Image Restoration Through Controlled Perturbation of Koschmieder's Model," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16205-16215, 2021.
15	Guo. C, Li. C, Guo. J, et al, "Zero-reference deep curve estimation for low-light image enhancement," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780-1789, 2020.
16	Jaffe. J. S, "Computer modeling and the design of optimal underwater imaging systems," IEEE Journal of Oceanic Engineering, vol. 15, no. 2, pp. 101-111. 1990. DOI
17	Fabbri. C, Islam. M. J, Sattar. J, "Enhancing underwater imagery using generative adversarial networks," in Proc. of 2018 IEEE International Conference on Robotics and Automation, pp. 7159-7165, 2018.
18	Li. C, Cong. R, Hou. J, et al, "Nested network with two-stream pyramid for salient object detection in optical remote sensing images," IEEE Transactions on Geoscience and Remote Sensing, vol. 57, pp. 11, pp. 9156-9166, 2019. DOI
19	Li. C, Guo. J, Guo. C, "Emerging from water: Underwater image color correction based on weakly supervised color transfer," IEEE Signal processing letters, vol. 25, no. 3, pp. 323-327, 2018. DOI
20	Ren. W, Liu. S, Ma. L, et al, "Low-light image enhancement via a deep hybrid network," IEEE Transactions on Image Processing, vol. 28, no. 9, pp. 4364-4375, 2019. DOI
21	Li. J, Wang. N, Zhang. L, et al, "Recurrent Feature Reasoning for Image Inpainting," in Proc. of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7760-7768, 2020.
22	Li. J, Skinner. K. A, Eustice. R. M, et al, "WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images," IEEE Robotics and Automation letters, vol. 3, no. 1, pp. 387-394, 2018. DOI
23	Hou. M, Liu. R, Fan. X, et al, "Joint residual learning for underwater image enhancement," in Proc. of 2018 25th IEEE International Conference on Image Processing, pp. 4043-4047, 2018.
24	Peng. Y. T, Cosman. P. C, "Underwater image restoration based on image blurriness and light absorption," IEEE transactions on image processing, vol. 26, no. 4, pp. 1579-1594, 2017. DOI
25	Akkaynak. D, Treibitz, T, "Sea-thru: A method for removing water from underwater images," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1682-1691, 2019.
26	Marques. T. P, Albu. A. B, "L² uwe: A framework for the efficient enhancement of low-light underwater images using local contrast and multi-scale fusion," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 538-539, 2020.
27	Uplavikar. P. M, Wu. Z, Wang. Z, "All-in-One Underwater Image Enhancement Using Domain-Adversarial Learning," in Proc. of CVPR Workshops, pp. 1-8, 2019.
28	Jamadandi. A, Mudenagudi. U, "Exemplar-based underwater image enhancement augmented by wavelet corrected transforms," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 11-17, 2019.
29	Desai. C, Tabib. R. A, Reddy. S. S, et al, "RUIG: Realistic Underwater Image Generation Towards Restoration," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 2181-2189, 2021.
30	Deng. J, Dong. W, Socher. R, et al, "Imagenet: A large-scale hierarchical image database," in Proc. of 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, 2009.
31	Chen. X, Zhang. P, Quan. L, et al, "Underwater Image Enhancement based on Deep Learning and Image Formation Model," arXiv:2101.00991v2, 2021.
32	Liu. X, Gao. Z, Chen. B. M, "MLFcGAN: Multilevel feature fusion-based conditional GAN for underwater image color correction," IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 9, pp. 1488-1492, 2020. DOI
33	Huang. D, Wang. Y, Song. W, et al, "Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition," in Proc. of International conference on multimedia modeling, Springer, Cham, pp. 453-465, 2018.
34	Li. C. Y, Guo. J. C, Cong. R. M, et al, "Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior," IEEE Transactions on Image Processing, vol. 25, no. 12, pp. 5664-5677, 2016. DOI
35	Sousa. J. P, Ferreira. B. M, Cruz. N. A, "Guidance of an autonomous surface vehicle for underwater navigation Aid," in Proc. of 2018 IEEE/OES Autonomous Underwater Vehicle Workshop, pp. 1-6, 2018.
36	Petillot. Y. R, Antonelli. G, Casalino. G, et al, "Underwater robots: From remotely operated vehicles to intervention-autonomous underwater vehicles," IEEE Robotics & Automation Magazine, vol. 26, no. 2, pp. 94-101, 2019.
37	Zhang. Z, Wu. Y, "Adaptive fuzzy tracking control of autonomous underwater vehicles with output constraints," IEEE Transactions on Fuzzy Systems, vol. 29, no. 5, pp. 1311-1319, 2021. DOI
38	Hu. H, Zhao. L, Li. X, et al, "Underwater image recovery under the nonuniform optical field based on polarimetric imaging," IEEE Photonics Journal, vol. 10, no. 1, pp. 1-9, 2018.
39	Akkaynak. D, Treibitz. T, "A revised underwater image formation model," in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 6723-6732, 2018.
40	Sanchez-Ferreira. C, Coelho. L. S, Ayala. H. V. H, et al, "Bio-inspired optimization algorithms for real underwater image restoration," Signal Processing: Image Communication, vol. 77, no. 1, pp. 49-65, 2019. DOI