KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Perceptual Generative Adversarial Network for Single Image De-Snowing

단일 영상에서 눈송이 제거를 위한 지각적 GAN

  • Received : 2019.07.05
  • Accepted : 2019.08.18
  • Published : 2019.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Image de-snowing aims at eliminating the negative influence by snow particles and improving scene understanding in images. In this paper, a perceptual generative adversarial network based a single image snow removal method is proposed. The residual U-Net is designed as a generator to generate the snow free image. In order to handle various sizes of snow particles, the inception module with different filter kernels is adopted to extract multiple resolution features of the input snow image. Except the adversarial loss, the perceptual loss and total variation loss are employed to improve the quality of the resulted image. Experimental results indicate that our method can obtain excellent performance both on synthetic and realistic snow images in terms of visual observation and commonly used visual quality indices.

눈이 내리는 영상에서 눈송이들에 의하여 영상의 질이 저하되고 영상 내에 존재하는 객체들을 명확히 탐지하기 위해서는 눈송이를 제거해야할 필요성이 있다. 이 연구에서는 지각 Generative Adversarial Network에 기반하여 단일 영상으로부터 눈송이를 제거하는 방법을 제시한다. 잔류 U-Net을 눈송이가 제거된 영상을 생성하는 생성기로 설계하였다. 다양한 크기의 눈송이를 처리하기 위하여 다양한 필터 커널의 인셉션 모듈을 설계하고 입력한 눈이 내리는 영상의 다양한 해상도 특징을 추출하기 위하여 적용되었다. 눈송이 제거 영상의 품질을 높이기 위해서 대립손실을 제외하고는, 지각적 손실과 총 변동 손실 함수를 적용하여 제설 이미지와의 유사도를 찾아갈 수 있도록 하였다. 합성 강설 이미지와 실제 강설 이미지를 대상으로 제안 네크워크의 제설 기능을 실험하였다. 실험 결과 제안 알고리즘은 합성 이미지와 강설 이미지 모든 분야에서 육안으로 관찰해본 결과 화질이 우수함을 보여주었고, 객관적 평가를 위하여 신호강도를 나타내는 PSNR과 구조변화를 측정하는 SSIM 인덱스를 비교하였으며, 제안 알고리즘이 지수 상으로도 가장 우수한 성능을 보여주었다.

Keywords

References

  1. X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley, "Lightweight pyramid networks for image deraining," arXiv preprint arXiv: 1805.06173, 2018.
  2. L. Deng, T. Huang, X. Zhao, and T. Jiang, "A directional global sparse model for single image rain removal," Applied Mathematical Modelling, Vol.59, pp.662-679, 2018, https://doi.org/10.1016/j.apm.2018.03.001
  3. ImageAI [Internet], http://imageai.org/.
  4. R. Li. J. Pan, Z. Li, and J. Tang, "Single image dehazing via conditional generative adversarial network," in Proceedings of the CVPR, pp.8202-8211, 2018.
  5. W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo, and S. Yan, "Deep joint rain detection and removal from a single image," in Proceedings of the CVPR, pp.1357-1366, 2017.
  6. S. Pei, Y. Tsai, and C. Lee, "Removing rain and snow in a single image using saturation and visibility features," IEEE International Conference on Multimedia and Expo Workshops, pp.1-6, 2014.
  7. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," International Conference on Medical Image Computing and Computer-assisted Intervention, pp.234-241, 2015.
  8. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets. in International Conference on Neural Information Processing Systems," pp.2672-2680, 2014.
  9. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1-9, 2015:
  10. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceedings of the CVPR, pp.2472-2481, 2018.
  11. O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, "Deblurgan: Blind motion deblurring using conditional adversarial networks," in Proceedings of CVPR, pp.8183- 8192, 2018.
  12. Z. Fan, H. Wu, X. Fu, Y. Huang, and X. Ding, "Residualguide network for single image deraining," 2018 ACM Multimedia Conference on Multimedia, pp.1751-1759, 2018.
  13. B. Wu, H. Duan, Z. Liu, and G. Sun, "SRPGAN: Perceptual generative adversarial network for single image super resolution. arXiv preprint arXiv:1712.05927, 2017.
  14. J. Johnson, A. Alahi, L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution," European Conference on Computer Vision, pp.694-711, 2016.
  15. Y. F. Liu, D. W. Jaw, S. C. Huang, and J. N. Hwang, "DesnowNet: Context-aware deep network for snow removal," IEEE Transactions on Image Processing, Vol.27, No.6, pp.3064-3073, 2018. https://doi.org/10.1109/TIP.2018.2806202
  16. X. Zheng, Y. Liao, W. Guo, X. Fu, and X. Ding, "Singleimage-based rain and snow removal using multi-guided filter," in Proc. Int. Conf. Neural Inf. Process, pp.258-265, 2013.
  17. Z. Wang, A. C. Bovik, H. Sheikh, and E. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, Vol.13, No.4, pp.600-612, 2004. https://doi.org/10.1109/TIP.2003.819861