Browse > Article
http://dx.doi.org/10.3837/tiis.2021.12.009

Lightweight multiple scale-patch dehazing network for real-world hazy image  

Wang, Juan (Hubei Energy Internet Engineering Technology Research Center, Hubei University of Technology)
Ding, Chang (Hubei Energy Internet Engineering Technology Research Center, Hubei University of Technology)
Wu, Minghu (Hubei Energy Internet Engineering Technology Research Center, Hubei University of Technology)
Liu, Yuanyuan (Hubei Energy Internet Engineering Technology Research Center, Hubei University of Technology)
Chen, Guanhai (Hubei Energy Internet Engineering Technology Research Center, Hubei University of Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.12, 2021 , pp. 4420-4438 More about this Journal
Abstract
Image dehazing is an ill-posed problem which is far from being solved. Traditional image dehazing methods often yield mediocre effects and possess substandard processing speed, while modern deep learning methods perform best only in certain datasets. The haze removal effect when processed by said methods is unsatisfactory, meaning the generalization performance fails to meet the requirements. Concurrently, due to the limited processing speed, most dehazing algorithms cannot be employed in the industry. To alleviate said problems, a lightweight fast dehazing network based on a multiple scale-patch framework (MSP) is proposed in the present paper. Firstly, the multi-scale structure is employed as the backbone network and the multi-patch structure as the supplementary network. Dehazing through a single network causes problems, such as loss of object details and color in some image areas, the multi-patch structure was employed for MSP as an information supplement. In the algorithm image processing module, the image is segmented up and down for processed separately. Secondly, MSP generates a clear dehazing effect and significant robustness when targeting real-world homogeneous and nonhomogeneous hazy maps and different datasets. Compared with existing dehazing methods, MSP demonstrated a fast inference speed and the feasibility of real-time processing. The overall size and model parameters of the entire dehazing model are 20.75M and 6.8M, and the processing time for the single image is 0.026s. Experiments on NTIRE 2018 and NTIRE 2020 demonstrate that MSP can achieve superior performance among the state-of-the-art methods, such as PSNR, SSIM, LPIPS, and individual subjective evaluation.
Keywords
Image enhancement; Convolutional neural networks; Image processing;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Qin, Z. L. Wang, Y. C. Bai, X. D. Xie, and H. Z.Jia, "FFA-Net: Feature fusion attention network for single image dehazing," in Proc. of the AAAI Conference on Artificial Intelligence, vol. 34(07), pp. 11908-11915, 2020.
2 R. T. Li, X. Y. Zhang, S. D. You, Y. Li, "Learning to Dehaze From Realistic Scene with A Fast Physics Based Dehazing Network," arXiv preprint arXiv:2004.08554, 2020.
3 Yang, X., Xu, Z., Luo, J., "Towards Perceptual Image Dehazing by Physics-Based Disentanglement and Adversarial Training," in Proc. of the AAAI Conference on Artificial Intelligence, vol. 32(1), 2018.
4 D. Chen et al., "Gated Context Aggregation Network for Image Dehazing and Deraining," in Proc. of 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375-1383, 2019.
5 D. Engin, A. Genc, H. K. Ekenel, "Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 825-833, 2018.
6 J. Y. Zhu, T. Park, P. Isola, A. A. Efros, "Unpaired Image-To-Image Translation Using CycleConsistent Adversarial Networks," in Proc. of the IEEE International Conference on Computer Vision (ICCV), pp. 2242 - 2251, 2017.
7 Zhang, Y. F., Li D., and G. Sharma, "Hazerd: an outdoor scene dataset and benchmark for single image dehazing," in Proc. of 2017 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 3205-3209, 2017.
8 S. Zhao, L. Zhang, S. Huang, Y. Shen and S. Zhao, "Dehazing Evaluation: Real-World Benchmark Datasets, Criteria, and Baselines," IEEE Transactions on Image Processing, vol. 29, pp. 6947-6962, 2020.   DOI
9 A. Gaidon, Q. Wang, Y. Cabon, E. Vig, "Virtual worlds as proxy for multi-object tracking analysis," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4340-4349, 2016.
10 Ancuti C., Ancuti C.O., Timofte R., De Vleeschouwer C, "I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images," in Proc. of International Conference on Advanced Concepts for Intelligent Vision Systems, Springer, Cham, 2018.
11 J. Zhang, Y. Cao, Z. J. Zha, D. C. Tao, "Nighttime Dehazing with a Synthetic Benchmark," in Proc. of the 28th ACM International Conference on Multimedia, 2020.
12 Y. Y. Qu, Y. Z. Chen, J. Y. Huang, Y. Xie, "Enhanced Pix2pix Dehazing Network," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8152-8160, 2019.
13 B. Li, X. Peng, Z. Wang, J. Xu, and D. Feng, "AOD-Net: All-in-one dehazing network," in Proc. of the IEEE International Conference on Computer Vision, vol. 1, p. 7, 2017.
14 C.O. Ancuti, C. Ancuti, R. Timofte, "NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 1798-1805, 2020.
15 Ronneberger O., Fischer P., Brox T, "U-Net: Convolutional Networks for Biomedical Image Segmentation," In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention (MICCAI). vol 9351, pp. 234-241, 2015.
16 S. Zhao, L. Zhang, Y. Shen and Y. Zhou, "RefineDNet: A Weakly Supervised Refinement Framework for Single Image Dehazing," in Proc. of IEEE Transactions on Image Processing (TIP), vol. 30, pp. 3391-3404, 2021.
17 K. He, J. Sun and X. Tang, "Single Image Haze Removal Using Dark Channel Prior," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2341-2353, Dec. 2011.   DOI
18 A Benoit, Leonel Cuevas, Jean-Baptiste Thomas, "Deep learning for dehazing: Comparison and analysis," in Proc. of Colour and Visual Computing Symposium (CVCS), 2018.
19 S. D. Das, S.t Dutta, "Fast Deep Multi-Patch Hierarchical Network for Nonhomogeneous Image Dehazing," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 482-483, 2020.
20 Y.J. Shao, L. Li, W. Q. Ren, C. X. Gao, N. Sang, "Domain Adaptation for Image Dehazing," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2805-2814, 2020.
21 B. Cai, X. Xu, K. Jia, C. Qing and D. Tao, "DehazeNet: An End-to-End System for Single Image Haze Removal," IEEE Transactions on Image Processing, vol. 25, no. 11, pp. 5187-5198, Nov. 2016.   DOI
22 Mccartney, E. J., "Scattering phenomena. (book reviews: Optics of the atmosphere. scattering by molecules and particles)," Science, 196, 1084-1085, 1977.   DOI
23 R. T. Tan, "Visibility in bad weather from a single image," in Proc. of 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008.
24 Q. Zhu, J. Mai and L. Shao, "A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior," IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3522-3533, Nov. 2015.   DOI
25 Zhang, H., Patel, V. M., Patel, V. M. and Patel, V. M, "Densely connected pyramid dehazing network," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 3194-3203, 2018.
26 X.H. Liu, Y.R. Ma, Z.H. Shi and J. Chen, "GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing," in Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7314-7323, 2019.
27 W.Q. Ren, L. Ma, J.W. Zhang, J.S. Pan, X.C. Cao, W. Liu, M.H. Yang, "Gated Fusion Network for Single Image Dehazing," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3253-3261, 2018.
28 Y. Dong, Y. Liu, H. Zhang, S. Chen, Y. Qiao, "FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing," in Proc. of the AAAI Conference on Artificial Intelligence, vol. 34(07), pp. 10729-10736. 2020.
29 T. T. Guo, V. Cherukuri, and V. Monga, "Dense '123' color enhancement dehazing network," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2131-2139, 2019.
30 T. T. Guo, X. L. Li, V. Cherukuri, and V. Monga, "Dense scene information estimation network for dehazing," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 2122-2130, 2019.
31 H. Zhu, X. Peng, V. Chandrasekhar, L. Li, J.H. Lim, "DehazeGAN: When Image Dehazing Meets Differential Programming," IJCAI, 2018.
32 W. Wang, A. Wang, Q. Ai, C. Liu and J. Liu, "AAGAN: Enhanced Single Image Dehazing With Attention-to-Attention Generative Adversarial Network," IEEE Access, vol. 7, pp. 173485-173498, 2019.   DOI
33 N. Wang, Y.B. Zhou, F.L. Han, H.Y. Zhu, Y.J. Zheng, "UWGANUnderwater GAN for Real-world Underwater Color," arXiv:1912.10269. 2019.
34 H. G. Zhang, Y. C. Dai, H. D. Li, and P. Koniusz, "Deep stacked hierarchical multi-patch network for image deblurring," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5971-5979, 2019.
35 J. Zhang, Y. Cao, and Z.F. Wang, "Nighttime haze removal based on a new imaging model," in Proc. of 2014 IEEE International Conference on Image Processing (ICIP), pp. 4557-4561, 2014.
36 Y. Li, R. T. Tan, and M. S. Brown, "Nighttime haze removal with glow and multiple light colors," in Proc. of the IEEE International Conference on Computer Vision (ICCV), pp. 226-234, 2015.
37 A. Mehta, H. Sinha, P. Narang, M. Mandal, "HIDEGAN: A Hyperspectral-guided Image Dehazing GAN," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 846-856, 2020.
38 Y. K. Yu, H. Liu, M. H. Fu, J. Chen, X. Y. Wang, K. Y. Wang, "A Two-Branch Neural Network for Non-Homogeneous Dehazing via Ensemble Learning," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 193-202, 2021.
39 H. Y. Wu, Y. Y. Qu, S. H. Lin, J. Zhou, R. Z. Qiao, Z. Z. Zhang, Y. Xie, L. Z. Ma, "Contrastive Learning for Compact Single Image Dehazing," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10551-10560, 2021.
40 R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586-595, 2018.
41 C. O. Ancuti, C. Ancuti, R. Timofte, C.D. Vleeschouwer, "O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 754-762, 2016.
42 H. Dong, J.S. Pan, L. Xiang, Z. Hu, X.Y. Zhang, F. Wang, M.H. Yang, "Multi-Scale Boosted Dehazing Network With Dense Feature Fusion," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2154-2167, 2020.
43 J. Zhang, Y. Cao, S. Fang, Y. Kang, and C. W. Chen, "Fast haze removal for nighttime image using maximum reflectance prior," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7418-7426, 2017.
44 Ancuti, C., Ancuti, C. O., and Timofte, R., "Ntire 2018 challenge on imagedehazing: Methods and results," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 891-901, 2018.
45 B. Li, W. Ren, D. Fu, D. Tao, D. Feng, W. Zeng, and Z. Wang, "Benchmarking single-image dehazing and beyond," IEEE Transactions on Image Processing (TIP), vol. 28(1), pp. 492-505, 2019.   DOI
46 C. Ancuti, C. O. Ancuti, and C. De Vleeschouwer, "D-HAZY: a dataset to evaluate quantitatively dehazing algorithms," in Proc. of IEEE International Conference on Image Processing (ICIP), IEEE, pp. 2226-2230, 2016.