인터넷정보학회논문지 (Journal of Internet Computing and Services)

  • 제19권5호
  • /
  • Pages.43-54
  • /
  • 2018
  • /
  • 1598-0170(pISSN)
  • /
  • 2287-1136(eISSN)
한국인터넷정보학회 (Korean Society for Internet Information)
권호 표지
DOI QR코드

DOI QR Code

생성적 대립쌍 신경망을 이용한 깊이지도 기반 연무제거

Single Image Dehazing Based on Depth Map Estimation via Generative Adversarial Networks

  • Wang, Yao (Department of Computer Science and Engineering, Hanyang University) ;
  • Jeong, Woojin (Department of Computer Science and Engineering, Hanyang University) ;
  • Moon, Young Shik (Department of Computer Science and Engineering, Hanyang University)
  • 투고 : 2018.06.15
  • 심사 : 2018.09.10
  • 발행 : 2018.10.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

연무가 있는 상황에서 촬영된 영상은 낮은 대비로 인해 시인성이 낮아지는 문제가 있다. 이렇게 연무로 인해 흐릿한 영상에서 연무의 효과를 제거하는 과정을 연무제거라고 한다. 연무제거에서 가장 중요한 문제 중 하나는 전달지도 (transmission map) 또는 깊이지도 (depth map)를 정확하게 추정하는 것이다. 본 논문에서는 정확한 깊이지도 추정을 위해 생성적 대립쌍 신경망 (Generative Adversarial Network: GAN)을 이용한 정확한 깊이 영상 추정 방법을 제안한다. 제안된 GAN 모델은 흐릿한 입력영상과 이에 상응하는 깊이지도 간의 비선형 매핑을 학습한다. 그리고 연무제거단계에서는 훈련된 모델을 사용하여 입력영상의 깊이지도를 추정하고 이것을 전달지도를 계산하는데 사용한다. 이어서 guided filter를 사용하여 전달지도를 다듬는다. 마지막으로 대기 산란 모델을 기반으로 연무가 제거된 영상을 복원한다. 제안된 GAN 모델은 합성실내영상으로 훈련되었다. 하지만 실제 연무영상에 대해서도 적용할 수 있다. 이를 실험을 통해 증명하였다. 또한 실험에서 제안된 방법이 이전에 연구된 방법에 비해 시각적 및 정량적 측면에서 우수한 결과를 나타냈다.

Images taken in haze weather are characteristic of low contrast and poor visibility. The process of reconstructing clear-weather image from a hazy image is called dehazing. The main challenge of image dehazing is to estimate the transmission map or depth map for an input hazy image. In this paper, we propose a single image dehazing method by utilizing the Generative Adversarial Network(GAN) for accurate depth map estimation. The proposed GAN model is trained to learn a nonlinear mapping between the input hazy image and corresponding depth map. With the trained model, first the depth map of the input hazy image is estimated and used to compute the transmission map. Then a guided filter is utilized to preserve the important edge information of the hazy image, thus obtaining a refined transmission map. Finally, the haze-free image is recovered via atmospheric scattering model. Although the proposed GAN model is trained on synthetic indoor images, it can be applied to real hazy images. The experimental results demonstrate that the proposed method achieves superior dehazing results against the state-of-the-art algorithms on both the real hazy images and the synthetic hazy images, in terms of quantitative performance and visual performance.

키워드

참고문헌

  1. D. Nan, D.-Y. Bi, L.-Y. He, S.-P. Ma and Z.-L. Fan, "A Variational Framework for Single Image Dehazing Based on Restoration," KSII Transactions on Internet and Information Systems, vol. 10, no. 3, pp. 1182-1194, 2016. http://dx.doi.org/10.3837/tiis.2016.03.013
  2. S. G. Narasimhan and S. K. Nayar, "Chromatic framework for vision in bad weather,"Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2000. https://doi.org/10.1109/CVPR.2000.855874
  3. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, "Instant dehazing of images using polarization," Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2001. https://doi.org/10.1109/CVPR.2001.990493
  4. J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, and D. Lischinski, "Deep photo: Model-based photograph enhancement and viewing" Proceedings ACM Transactions on Graphics, vol. 27, 2008. https://doi.org/10.1145/1457515.1409069
  5. K. He, J. Sun, and X. Tang, "Single image haze removal using dark channel prior," Proceedings IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, pp. 2341-2353, 2010. https://doi.org/10.1109/TPAMI.2010.168
  6. X. Zhou, C. Wang, L. Wang, N. Wang, and Q. Fu "Single Image Dehazing Using Dark Channel Prior and Minimal Atmospheric Veil,"KSII Transactions on Internet and Information Systems, vol. 10, no. 1, pp. 341-363, 2016. http://dx.doi.org/10.3837/tiis.2016.01.020
  7. L. Wang, X. Zhou, C. Wang and W. Li, "The Effects of Image Dehazing Methods Using Dehazing Contrast-Enhancement Filters on Image Compression," KSII Transactions on Internet and Information Systems, vol. 10, no. 7, pp. 3245-3271, 2016. http://dx.doi.org/10.3837/tiis.2016.07.021
  8. Q. Zhu, J. Mai, and L. Shao, "Single Image Dehazing Using Color Attenuation Prior," British Machine Vision Conference, 2014. https://doi.org/10.1109/TIP.2015.2446191
  9. W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, and MH. Yang, "Single image dehazing via multi-scale convolutional neural networks," Proceedings European Conference on Computer Vision, pp. 154-169, 2016. https://doi.org/10.1007/978-3-319-46475-6_10
  10. B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, "DehazeNet: An End-to-End System for Single Image Haze Removal," Proceedings IEEE Transactions on Image Processing, vol. 25, pp. 5187-5198, 2016. https://doi.org/10.1109/TIP.2016.2598681
  11. H. Koschmieder, "Theorie der horizontalen Sichtweite," Beitrage zur Physik der freien Atmosphare, vol. 640, pp. 7-10, 1959. https://doi.org/10.1007/978-3-663-04661-5_2
  12. I. Goodfellow, J. P-Abadie, M. Mirza, B. Xu, D. W-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," Advances in Neural Information Processing Systems Conference, 2014. https://dl.acm.org/citation.cfm?id=2969125
  13. Z. Yi, H. Zhang, P. Tan, and M. Gong, "DualGAN: Unsupervised Dual Learning for Image-to-Image Translation," IEEE International Conference on Computer Vision, pp. 2868-2876, 2017. http://doi.org/10.1109/ICCV.2017.310
  14. P. L. Suarez, A. D. Sappa, and B. X. Vintimilla, "Infrared image colorization based on a triplet DCGAN architecture," IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 212-217, 2017. https://doi.org/10.1109/CVPRW.2017.32
  15. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, and W. Shi, "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2017. http://dx.doi.org/10.1109/CVPR.2017.19
  16. S. Shwartz, E. Namer, and Y. Y. Schechner, "Blind haze separation." Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 2006. https://doi.org/10.1109/CVPR.2006.71
  17. S. G. Narasimhan, and S. K. Nayar, "Interactive (de) weathering of an image using physical models," Proceedings IEEE Workshop on color and photometric Methods in computer Vision, vol. 6, 2003. https://www.ri.cmu.edu/publications/interactive-deweathering-of-an-image-using-physical-models/
  18. R. Fattal, "Single image dehazing," Proceedings ACM Transactions on Graphics, vol. 27, 2008. https://doi.org/10.1145/1360612.1360671
  19. R. T. Tan, "Visibility in bad weather from a single image," Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008. https://doi.org/10.1109/CVPR.2008.4587643
  20. G. Meng, Y. Wang, J. Duan, S. Xiang, and C. Pan, "Efficient image dehazing with boundary constraint and contextual regularization," Proceedings IEEE International Conference on Computer Vision, pp. 617-624, 2013. https://doi.org/10.1109/ICCV.2013.82
  21. K. Tang, J. Yang, and J. Wang, "Investigating haze-relevant features in a learning framework for image dehazing," Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 2995-3000, 2014. https://doi.org/10.1109/CVPR.2014.383
  22. P. Isola, J. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," Computer Vision and Pattern Recognition, 2017. http://doi.ieeecomputersociety.org/10.1109/CVPR.2017.632
  23. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," Proceedings International Conference on Medical image computing and computer-assisted intervention, pp. 234-241, 2015. https://doi.org/10.1007/978-3-319-24574-4_28
  24. K. He, J. Sun, and X. Tang, "Guided Image Filtering," Proceedings European Conference on Computer Vision, pp. 1-14, 2010. https://doi.org/10.1109/TPAMI.2012.213
  25. A. Cosmin, O. A. Codruta, and D. V. Christophe, "D-hazy: a dataset to evaluate quantitatively dehazing algorithms," Proceedings IEEE International Conference on Image Processing, pp. 2226-2230, 2016. http://dx.doi.org/10.1109/ICIP.2016.7532754
  26. D. Scharstein, H. Hirschmller, Y. Kitajima, G. Krathwohl, N. Nesic, X. Wang, and P. Westling, "High-resolution stereo datasets with subpixel-accurate ground truth," Proceedings German Conference on Pattern Recognition, pp. 31-42, 2014. http://dx.doi.org/10.1007/978-3-319-11752-2_3
  27. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and support inference from rgbd images," Proceedings European Conference on Computer Vision, pp. 746-760, 2012. https://doi.org/10.1007/978-3-642-33715-4_54

피인용 문헌

  1. R2와 어텐션을 적용한 유넷 기반의 영상 간 변환에 관한 연구 vol.21, pp.4, 2018, https://doi.org/10.7472/jksii.2020.21.4.9