Journal of Ocean Engineering and Technology (한국해양공학회지)

Korean Society of Ocean Engineers (한국해양공학회)
권호 표지
DOI QR코드

DOI QR Code

Single Image-based Enhancement Techniques for Underwater Optical Imaging

  • Kim, Do Gyun (Department of control and instrument engineering, Korea Maritime and Ocean University) ;
  • Kim, Soo Mee (Maritime ICT R&D Center, Korea Institute of Ocean Science and Technology)
  • Received : 2020.06.05
  • Accepted : 2020.11.27
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Underwater color images suffer from low visibility and color cast effects caused by light attenuation by water and floating particles. This study applied single image enhancement techniques to enhance the quality of underwater images and compared their performance with real underwater images taken in Korean waters. Dark channel prior (DCP), gradient transform, image fusion, and generative adversarial networks (GAN), such as cycleGAN and underwater GAN (UGAN), were considered for single image enhancement. Their performance was evaluated in terms of underwater image quality measure, underwater color image quality evaluation, gray-world assumption, and blur metric. The DCP saturated the underwater images to a specific greenish or bluish color tone and reduced the brightness of the background signal. The gradient transform method with two transmission maps were sensitive to the light source and highlighted the region exposed to light. Although image fusion enabled reasonable color correction, the object details were lost due to the last fusion step. CycleGAN corrected overall color tone relatively well but generated artifacts in the background. UGAN showed good visual quality and obtained the highest scores against all figures of merit (FOMs) by compensating for the colors and visibility compared to the other single enhancement methods.

Keywords

Acknowledgement

This study was supported in part by research grants from Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2018R1D1A1B07049296), and by the project titled 'Development of the support vessel and systems for the offshore field test and evaluation of offshore equipment', funded by the Ministry of Oceans and Fisheries (MOF), Korea.

References

  1. Achantay, R., Hemamiz, S., Estraday, F., & Susstrunk, S. (2009). Frequency Tuned Salient Region Detection. Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 1597-1604. https://doi.org/10.1109/CVPR.2009.5206596
  2. Ancuti, C.O., Ancuti, C., Vleeschouwer, C.D., & Bekaert, P. (2017). Color Balance and Fusion for Underwater Image Enhancement. IEEE Transactions on Image Processing, 27(1), 379-393. https://doi.org/10.1109/TIP.2017.2759252
  3. Anwar, S., Li, C., & Porikli, F. (2018). Deep Underwater Image enhancement. arXiv:1807.03528. Retrieved from https://arxiv.org/abs/1807.03528
  4. Anwar S. & Li, C. (2019). Diving Deeper into Underwater Image Enhancement: A Survey. arXiv:1907.07863. Retrieved 2019 from https://arxiv.org/abs/1907.07863
  5. Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein GAN. arXiv:1701.07875. Retrieved from https://arxiv.org/abs/1701.07875
  6. Arnold-Bos, A., Malkasse, J.-P., & Kervern, G. (2005). A Preprocessing Framework for Automatic Underwater Images Denoising. Proceeding of European Conference on Propagation and Systems, Brest, France, 1-8. Retrieved from https://hal.archives-ouvertes.fr/hal-00494314
  7. Bazeille, S., Quidu, I., Jaulin, L., & Malkasse, J.-P. (2006). Automatic Underwater Image Pre-processing. Proceeding of Caracterisation Du Milieu Marin, 16-19. Retrieved from https://hal.archivesouvertes.fr/hal-00504893/
  8. Berman, D., Levy, D., Avidan, S., & Treibitz, T. (2018). Underwater single Image Color Restoration Using Haze-lines and a New Quantitative Dataset. arXiv:1811.01343. Retrieved from https://arxiv.org/abs/1811.01343
  9. Cao, K., Peng, Y., & Cosman, P. (2018). Underwater Image Restoration Using Deep Networks to Estimate Background Light and Scene Depth. Proceeding of IEEE Southwest Symposium on Image Analysis and Interpretation, Las Vegas, NV, USA, 1-4. https://doi.org/10.1109/SSIAI.2018.8470347
  10. Carlevaris-Bianco, N., Mohan, A., & Eustice, R.M. (2010). Initial Results in Underwater Single Image Dehazing. Proceeding of Oceans 2010 MTS/IEEE Seattle, WA, USA, 1-8. https://doi.org/10.1109/OCEANS.2010.5664428
  11. Cho, Y., Jeong, J., Kim A. (2018). Model-assisted Multiband Fusion for Single Image Enhancement and Applications to Robot Vision. IEEE Robotics and Automation Letters, 30(4), 2822-2829. https://doi.org/10.1109/LRA.2018.2843127
  12. Crete-Roffet, F., Dolmière, T., Ladret, P., & Nicolas, M. (2007). The Blur Effect: Perception and Estimation with a New No-Reference Perceptual Blur Metric. Proceeding SPIE Electronic Imaging Symposium Conf Human Vision and Electronic Imaging, San Jose, USA, EI6492-16. Retrieved from https://hal.archivesouvertes.fr/hal-00232709
  13. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). Imagenet: A Large-scale Hierarchical Image Database. Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 248-255. https://doi.org/10.1109/CVPR.2009.5206848
  14. Drews, P., Nascimento, E., Moraes, F., Botelho, S., & Campos, M. (2013). Transmission Estimation in Underwater Single Images. Proceeding of IEEE International Conference on Computer Vision Workshops, Sydney, NSW, Australia, 825-830. https://doi.org/10.1109/ICCVW.2013.113
  15. Fabbri, C., Islam, M.J., & Sattar, J. (2018). Enhancing Underwater Imagery Using Generative Adversarial Networks. Proceeding of IEEE International Conference on Robotics and Automation, Brisbane, QLD, Australia, 7159-7165. https://doi.org/10.1109/ICRA.2018.8460552
  16. Fattal, R. (2008). Single Image Dehazing. ACM Transactions on Graphics, 27(3), 1-9. https://doi.org/10.1145/1360612.1360671
  17. Ghani, A.S.A., & Isa, N.A.M. (2014). Underwater Image Quality Enhancement Trough Composition of Dual-intensity Images and Rayleigh-stretching. SpringerPlus, 3, 757. https://doi.org/10.1186/2193-1801-3-757
  18. Ghani, A.S.A., & Isa, N.A.M. (2015). Underwater Image Quality Eenhancement Through Integrated Color Model with Rayleigh Distribution. Applied Soft Computing, 27, 219-230. https://doi.org/10.1016/j.asoc.2014.11.020
  19. Guo, Y., Li, H., & Zhuang, P. (2019). Underwater Image Enhancement Using a Multiscale Dense Generative Adversarial Network. IEEE Journal of Oceanic Engineering, 45(3), 862-870. https://doi.org/10.1109/JOE.2019.2911447
  20. He, K., Sun, J., & Tang, X. (2011). Single Image Haze Removal Using Dark Channel Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2341-2353. https://doi.org/10.1109/TPAMI.2010.168
  21. Hou, M., Liu, R., Fan, X., & Luo, Z. (2018). Joint Residual Learning for Underwater Image Enhancement. Proceeding of IEEE International Conference on Image Processing, Athens, Greece, 4043-4047. https://doi.org/10.1109/ICIP.2018.8451209
  22. Isola, P., Zhu, J., Zhou, T., & Efros, A.A. (2017). Image-to-Image Translation with Conditional Adversarial Networks. arXiv: 1611.07004. Retrieved from https://arxiv.org/abs/1611.07004
  23. Jian S., & Wen, W. (2017). Study on Underwater Image Denoising Agorithm Based on Wavelet Transform. Journal of Physics: Conference Series, 806, 012006. Retrieved from https://iopscience.iop.org/article/10.1088/1742-6596/806/1/012006
  24. Khan, A., Ali, S.S.A., Malik, A., Anwer, A., & Meriaudeau, F. (2016). Underwater Image Enhancement by Wavelet Based Fusion. Proceeding of IEEE International Conference on Underwater System Technology: Theory and Applications, Penang, Malaysia, 83-88. https://10.1109/USYS.2016.7893927
  25. Li, W.J., Gu, B., Huang, J.T., Wang, S.Y., & Wang, M.H. (2012). Single Image Visibility Enhancement in Gradient Domain. IET Image Processing, 6(5), 589-595. https://doi.org//10.1049/iet-ipr.2010. 0574
  26. Li, C., Guo, C., & Guo, J. (2018a). Emerging from Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer. IEEE Signal Processing Letters, 25(3), 232-327. https://doi.org/10.1109/LSP.2018.2792050
  27. Li, J., Skinner, K., Eustice, R., & Johnson-Roberson, M. (2018b). Watergan: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater Images. IEEE Robotics and Automation Letters, 3(1), 387-394. https://doi.org/10.1109/LRA.2017.2730363.
  28. Li, C., Guo, C., Ren, W., Cong, R., Hou, J., & Kwong, S. (2019a). An Underwater Image Enhancement Dataset and Beyond. arXiv:1901.05495. Retrieved from https://arxiv.org/abs/1901.05495
  29. Li, H., Li, J., & Wang, W. (2019b). A Fusion Adversarial Underwater Image Enhancement Network with a Public Test Dataset. arXiv:1906.06819. Retrieved from https://arxiv.org/abs/1906.06819
  30. Luo, M., Fang, Y., & Ge, Y. (2019). An Effective Underwater Image Enhancement Method Based on CLAHE-HF. Journal of Physics: Conference Series, 1237(3), 032009. Retrieved from https://iopscience.iop.org/article/10.1088/1742-6596/1237/3/032009
  31. Mathieu, M., Couprie, C., & LeCun, Y. (2015). Deep Multi-Scale Video Prediction Beyond Mean Square Error. arXiv:1511.05440. Retrieved from https://arxiv.org/abs/1511.05440
  32. Mi, Z., Zhou, H., Zheng, Y., & Wang, M. (2016). Single Image Dehazing via Multiscale Gradient Domain Contrast Enhancement. IET Image Processing, 10(3), 206-214. https://doi.org/10.1049/iet-ipr.2015.0112
  33. Mobley, C. (1994). Light and Water: Radiative Transfer in Natural Waters. San Diego:Academic Press.
  34. Panetta, K., Agaian, S., Zhou, Y., & Wharton, E.J. (2011). Parameterized Logarithmic Framework for Image Enhancement. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 41(2), 460-473. https://doi.org/10.1109/TSMCB.2010.2058847
  35. Panetta, K., Gao, C., & Agaian, S. (2016). Human-Visual-SystemInspired Underwater Image Quality Measures. IEEE Journal of Oceanic Engineeringm, 41(3), 541-551. https://doi.org/10.1109/JOE.2015.2469915
  36. Park, E.P., & Sim, J. (2017). Gradient-based Contrast Enhancement and Color Correction for Underwater Images. Proceeding of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Kuala Lumpur, Malaysia, 1444-1447. https://doi.org/10.1109/APSIPA.2017.8282259
  37. Peng, Y., & Cosman, P.C. (2017). Underwater Image Restoration Based on Image Blurriness and Light Absorption. IEEE Transactions on Image Processing, 26(4), 1579-1594. https://doi.org/10.1109/TIP.2017.2663846
  38. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional Networks for Biomedical Image Segmentation. arXiv: 1505.04597. Retrieved from https://arxiv.org/abs/1505.04597
  39. Roser, M., Dunbabin, M., & Geiger, A. (2014). Simultaneous Underwater Visibility Assessment, Enhancement and Improved Stereo. Proceeding of IEEE International Conference on Robotics and Automation, Hong Kong, China, 3840-3847. https://doi.org/10.1109/ICRA.2014.6907416
  40. Simchony, T., Chellappa R., & Shao, M. (1990). Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(5), 435-446. https://doi.org/10.1109/34.55103
  41. Sun, X., Liu, L., Li, Q., Dong, J., Lima, E., & Yin, R. (2018). Deep Pixel to Pixel Network for Underwater Image Enhancement and Restoration. IET Image Processing, 13(3), 469-474. https://doi.org/10.1049/iet-ipr.2018.5237
  42. Tarel, J., & Hautière, N. (2009). Fast Visibility Restoration from a Single Color or Gray Level Image. Proceeding of IEEE International Conference on Computer Vision, Kyoto, Japan, 2201-2208. https://doi.org/10.1109/ICCV.2009.5459251
  43. Treibitz, T., & Schechner, Y.Y. (2012). Turbid Scene Enhancement Using Multi-Directional Illumination Fusion. IEEE Transactions on Image Processing, 21(11), 4662-4667. https://doi.org/10.1109/TIP.2012.2208978
  44. Uplavikar, P., Wu, Z., & Wang, Z. (2019). All-in-One Underwater Image Enhancement Using Domain-Adversarial Learning. arXiv: 1905.13342. Retrieved from https://arxiv.org/abs/1905.13342
  45. Vincent, L. (1993). Morphological Grayscale Reconstruction in Image Analysis: Applications and Efficient Algorithms. IEEE Transactions on Image Processing, 2(2), 176-201. https://doi.org/10.1109/83.217222
  46. Wang, Y., Ding, X., Wang, R., Zhang, J., & Fu, X. (2017a). Fusion-Based Underwater Image Enhancement by Wavelet Decomposition. Proceeding of IEEE International Conference on Industrial Technology, Toronto, ON, Canada, 1013-1018. https://doi.org/10.1109/ICIT.2017.7915500
  47. Wang, Y., Zhang, J., Cao, Y., & Wang, Z. (2017b). A Deep CNN Method for Underwater Image Enhancement. Proceeding of IEEE International Conference on Image Processing, Beijing, China, 1382-1386. https://doi.org/10.1109/ICIP.2017.8296508
  48. Wang, Y., Song, W., Fortino, G., Qi, L.-Z., Zhang, W., & Liotta, A. (2019). An Experimental-Based Review of Image Enhancement and Image Restoration Methods for Underwater Imaging. IEEE Access, 7, 140233-140251. https://doi.org/10.1109/ACCESS.2019.2932130
  49. Yang, M., & Sowmya, A. (2015). An Underwater Color Image Quality Evaluation Metric. IEEE Transactions on Image Processing, 24(12), 6062-6071. https://doi.org/10.1109/TIP.2015.2491020
  50. Yang, M., Hu, J., Li, C., Rohde, G., Du, Y., & Hu, K. (2019). An In-Depth Survey of Underwater Image Enhancement and Restoration. IEEE Access, 7, 123638-123657. https://doi.org/10.1109/ACCESS.2019.2932611
  51. Zetian, M., Zheng, L., Yafei, W., Xianping, F., & Zhengyu, C. (2018). Multiscale Gradient Domain Underwater Image Enhancement. Proceeding of 2018 Oceans - MTS/IEEE Kobe Techno-Oceans, Kobe, Japan, 1-5. https://doi.org/10.1109/OCEANSKOBE.2018.8559180
  52. Zhao, X., Tao, J., & Song, Q. (2015). Deriving Inherent Optical Properties from Background Color and Underwater Image Enhancement. Ocean Engineering, 94, 163-172. https://doi.org/10.1016/j.oceaneng.2014.11.036
  53. Zhang, S., Zhang, J., Fang, S., & Cao, Y. (2014). Underwater Stereo Image Enhancement Using a New Physical Model. Proceeding of IEEE International Conference on Image Processing, Paris, France, 5422-5426. https://doi.org/10.1109/ICIP.2014.7026097
  54. Zhu, J., Park, T., Isola, P., & Efros, A.A. (2018). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. arXiv:1703.10593. Retrieved from https://arxiv.org/abs/1703.10593