Browse > Article
http://dx.doi.org/10.9717/kmms.2019.22.9.1000

Deep Multi-task Network for Simultaneous Hazy Image Semantic Segmentation and Dehazing  

Song, Taeyong (School of Electrical Electronic Engineering, Yonsei University)
Jang, Hyunsung (EO/IR R&D Lab., LIG Nex1 Co., Ltd.)
Ha, Namkoo (EO/IR R&D Lab., LIG Nex1 Co., Ltd.)
Yeon, Yoonmo (EO/IR R&D Lab., LIG Nex1 Co., Ltd.)
Kwon, Kuyong (EO/IR R&D Lab., LIG Nex1 Co., Ltd.)
Sohn, Kwanghoon (School of Electrical Electronic Engineering, Yonsei University)
Publication Information
Journal of Korea Multimedia Society / v.22, no.9, 2019 , pp. 1000-1010 More about this Journal
Abstract
Image semantic segmentation and dehazing are key tasks in the computer vision. In recent years, researches in both tasks have achieved substantial improvements in performance with the development of Convolutional Neural Network (CNN). However, most of the previous works for semantic segmentation assume the images are captured in clear weather and show degraded performance under hazy images with low contrast and faded color. Meanwhile, dehazing aims to recover clear image given observed hazy image, which is an ill-posed problem and can be alleviated with additional information about the image. In this work, we propose a deep multi-task network for simultaneous semantic segmentation and dehazing. The proposed network takes single haze image as input and predicts dense semantic segmentation map and clear image. The visual information getting refined during the dehazing process can help the recognition task of semantic segmentation. On the other hand, semantic features obtained during the semantic segmentation process can provide cues for color priors for objects, which can help dehazing process. Experimental results demonstrate the effectiveness of the proposed multi-task approach, showing improved performance compared to the separate networks.
Keywords
Semantic Segmentation; Dehazing; Deep Learning; Multi-task Learning;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J, Dean et al., "Tensorflow: A System for Large-scale Machine Learning," 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp. 265-283, 2016.
2 D. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," Proceeding of International Conference on Learning Representations, 2015.
3 B. Lee and S. Kwon, "Automatic Classification System of Tablets with Various Colors and Shapes," Journal of Korea Multimedia Society, Vol. 21, No. 6, pp. 659-666, 2018.   DOI
4 H. Lee, J. Sa, H. Shin, Y. Chung, D. Park, and H. Kim, "Separation of Occluding Pigs using Deep Learning-based Image Processing Techniques," Journal of Korea Multimedia Society, Vol. 22, No. 2, pp 136-145, 2019.   DOI
5 Q. Zhu, J. Mai, and L. Shao, "A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior," IEEE Transactions on Image Processing, Vol. 24, No. 11, pp. 3522-3533, 2015.   DOI
6 Y. Jeong, I. Ansari, J. Shim, and J. Lee, "A Car Plate Area Detection System Using Deep Convolution Neural Network," Journal of Korea Multimedia Society, Vol. 20, No. 8, pp. 1166-1174, 2017.   DOI
7 C. Oh, B. Kim, and K. Sohn, "Automatic Illumination Invariant Road Detection with Stereo Vision," Proceedings of the IEEE Conference on Industrial Electronics and Applications, pp. 889-893, 2012.
8 A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
9 S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks," Advances in Neural Information Processing Systems, pp 91-99, 2015.
10 K. He, J. Sun, and X. Tang, "Single Image Haze Removal Using Dark Channel Prior," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 12, pp. 2341-2353, 2011.   DOI
11 B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, "Dehazenet: An End-to-end System for Single Image Haze Removal," IEEE Transactions on Image Processing, Vol. 25, No. 11, pp. 5187-5198, 2016.   DOI
12 H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid Scene Parsing Network," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 6230-6239, 2016.
13 W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, and M.H. Yang, "Single Image Dehazing via Multi-scale Convolutional Neural Networks," Proceeding of European Conference on Computer Vision, pp. 156-169, 2016.
14 H. Zhang and V.M. Patel, "Densely Connected Pyramid Dehazing Network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3194-3203, 2018.
15 J. Long, E. Shelhamer, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431-3440, 2015.
16 V. Badrinarayanan, A. Kendall, and R. Cipolla, "Segnet: A Deep Convolutional Encoder-decoder Architecture for Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495, 2017.   DOI
17 O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional Networks for Biomedical Image Segmentation," International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234-241, 2015.
18 C. Sakaridis, D. Dai, and L. Van Gool, "Semantic Foggy Scene Understanding with Synthetic Data," International Journal of Computer Vision, 2018, Vol. 126, No. 9, pp. 973-992.   DOI
19 D. Dai, C. Sakaridis, S. Hecker, and L. Van Gool, "Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding," International Journal of Computer Vision, pp. 1-23, 2019.
20 I. Goodfellow, D. Warde-Farley, M. Mirza, A. C. Courville, and Y. Bengio, "Maxout Networks," Proceedings of the 30th International Conference on Machine Learning, pp. 1319-1327, 2013.
21 W. Ren, J. Zhang, X. Xu, L. Ma, X. Cao, G. Meng, et al., "Deep Video Dehazing with Semantic Segmentation," IEEE Transactions on Image Processing, Vol. 28, No. 4, pp. 1895-1908, 2018.   DOI
22 G. Lin, A. Milan, C. Shen, and I. Reid, "Refinenet: Multipath Refinement Networks with Identity Mappings for High-resolution Semantic Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1925-1934, 2017.
23 L. Pan, Y. Dai, M. Liu, and F. Porikli, "Simultaneous Stereo Video Deblurring and Scene Flow Estimation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4382-4391, 2017.
24 T. Song, Y. Kim, C. Oh, and K. Sohn, "Deep Network for Simultaneous Stereo Matching and Dehazing," British Machine Vision Conference, 2018.
25 X. Qi, R. Liao, Z. Liu, R. Urtasun, and J. Jia, "Geonet: Geometric Neural Network for Joint Depth and Surface Normal Estimation," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 283-291, 2018.
26 X. Zhang, H. Dong, Z. Hu, W. Lai, F. Wang, and M. Yang, "Gated Fusion Network for Joint Image Deblurring and Super-resolution," British Machine Vision Conference, 2018.
27 K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-scale Image Recognition," Proceeding of International Conference on Learning Representations, 2015.
28 M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, et al., "The Cityscapes Dataset for Semantic Urban Scene Understanding," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213-3223, 2016.