IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Video Inpainting Performance using Denoising Technique

잡음 제거 기술 기반의 비디오 인페인팅 성능 연구

  • Received : 2022.10.19
  • Accepted : 2022.11.26
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we study the effect of noise on video inpainting, a technique that fills missing regions of video. Since the video may contain noise, the quality of the video may be affected when applying the video inpainting technique. Therefore, in this paper, we compare the inpainting performance in video with and without denoising techniqueDAVIS dataset. For that, we conducted two experiments: 1) applying inpainting technique after denoising the noisy video and 2) applying the inpainting technique to the video and denoising the video. Through the experiment, we observe the effect of denoising technique on the quality of video inpainting and conclude that video inpainting after denoising would improve the quality of the video subjectively and objectively.

Keywords

Acknowledgement

이 논문은 2020년도 정부 (교육부)의 재원으로 한국연구재단의 지원을 받아 수행된 기초연구사업임 (No. 2020R1I1A3072227).

References

  1. M. Bertalmio, L. Vese, G. Sapiro, S. Osher, "Simultaneous Structure and Texture Image Inpainting," IEEE Transactions on Image Processing, Vol. 12, No. 8, pp. 882-889, 2003. https://doi.org/10.1109/TIP.2003.815261
  2. S. Darabi, E. Shechtman, C. Barnes, D. Goldman, P. Sen, "Image Modeling: Combining Inconsistent Images Using Patch-based Synthesis," ACM Transactions on Graphics, Vol. 31, No. 4, Art. 82, pp. 1-10, 2012.
  3. M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester, "Image Inpainting," Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417-424, 2000.
  4. C. Ballester, M. Bertalmio, V. Caselles, G. Saprio, J. Verder, "Filling-in by Joint Interpolation of Vector Fields and Gray Levels," IEEE Transactions on Image Processing, Vol. 10, No. 8, pp. 1200-1211, 2001. https://doi.org/10.1109/83.935036
  5. G. Liu, F. A. Reda, K. J. Shih, T. C. Wang, A. Tao, B. Catanzaro, "Image Inpainting for Irregular Holes Using Partial Convolutions," Proceedings of the European Conference on Computer Vision, pp. 85-100, 2018.
  6. J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, T. Huang, "Free-form Image Inpainting with Gated Convolution," Proceedings of IEEE/CVF International Conference on Computer Vision, pp. 4470-4479, 2019.
  7. S. Iizuka, E. Simo-Serra, H. Ishikawa, "Globally and Locally Consistent Image Completion," ACM Transactions on Graphics, Vol. 36, No. 4, pp. 1-14, 2017.
  8. J. Yu, J. Yang, X. Shen, X. Lu, T. S. Huang, "Generative Image Inpainting with Contextual Attention," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505-5514, 2018.
  9. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, "Generative Adversarial Nets," Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
  10. R. Xu, X. Li, B. Zhou, C.C. Loy, "Deep Flow-guided Video Inpainting," Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition,  pp. 3718-3727, 2017.
  11. D. Kim, S. Woo, J. Y. Lee, I. S. Kweon, "Deep Video Inpainting," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5792-5801, 2019.
  12. E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox, "Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462-2470, 2017.
  13. T. W. Hui, X. Tang. C, C. C. Loy, "LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estmation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8981-8989, 2018.
  14. M. Tassano, J. Delon, T. Veit, "FastDVDnet: Towards Real-time Deep Video Denoising Without Flow Estimation," Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1351-1360, 2020.
  15. F. Perazzi, J. Pont-Tuset, B. McWilliams, L. V. Gool, M. Gross, A. Sorkine-Hornung, "A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 724-732, 2016.