Browse > Article
http://dx.doi.org/10.14372/IEMEK.2022.17.6.329

A Study on the Video Inpainting Performance using Denoising Technique  

Jeong-yun, Seo (Kyungpook National University)
Han-gyul, Baek (Kyungpook National University)
Sang-hyo, Park (Kyungpook National University)
Publication Information
IEMEK Journal of Embedded Systems and Applications / v.17, no.6, 2022 , pp. 329-335 More about this Journal
Abstract
In this paper, we study the effect of noise on video inpainting, a technique that fills missing regions of video. Since the video may contain noise, the quality of the video may be affected when applying the video inpainting technique. Therefore, in this paper, we compare the inpainting performance in video with and without denoising techniqueDAVIS dataset. For that, we conducted two experiments: 1) applying inpainting technique after denoising the noisy video and 2) applying the inpainting technique to the video and denoising the video. Through the experiment, we observe the effect of denoising technique on the quality of video inpainting and conclude that video inpainting after denoising would improve the quality of the video subjectively and objectively.
Keywords
Deep learning; Noise reduction; Video denoising; Video inpainting;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 J. Yu, J. Yang, X. Shen, X. Lu, T. S. Huang, "Generative Image Inpainting with Contextual Attention," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505-5514, 2018.
2 I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, "Generative Adversarial Nets," Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
3 R. Xu, X. Li, B. Zhou, C.C. Loy, "Deep Flow-guided Video Inpainting," Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition,  pp. 3718-3727, 2017.
4 D. Kim, S. Woo, J. Y. Lee, I. S. Kweon, "Deep Video Inpainting," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5792-5801, 2019.
5 E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox, "Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462-2470, 2017.
6 T. W. Hui, X. Tang. C, C. C. Loy, "LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estmation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8981-8989, 2018.
7 M. Tassano, J. Delon, T. Veit, "FastDVDnet: Towards Real-time Deep Video Denoising Without Flow Estimation," Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1351-1360, 2020.
8 F. Perazzi, J. Pont-Tuset, B. McWilliams, L. V. Gool, M. Gross, A. Sorkine-Hornung, "A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 724-732, 2016.
9 M. Bertalmio, L. Vese, G. Sapiro, S. Osher, "Simultaneous Structure and Texture Image Inpainting," IEEE Transactions on Image Processing, Vol. 12, No. 8, pp. 882-889, 2003.   DOI
10 S. Darabi, E. Shechtman, C. Barnes, D. Goldman, P. Sen, "Image Modeling: Combining Inconsistent Images Using Patch-based Synthesis," ACM Transactions on Graphics, Vol. 31, No. 4, Art. 82, pp. 1-10, 2012.
11 M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester, "Image Inpainting," Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417-424, 2000.
12 C. Ballester, M. Bertalmio, V. Caselles, G. Saprio, J. Verder, "Filling-in by Joint Interpolation of Vector Fields and Gray Levels," IEEE Transactions on Image Processing, Vol. 10, No. 8, pp. 1200-1211, 2001.   DOI
13 G. Liu, F. A. Reda, K. J. Shih, T. C. Wang, A. Tao, B. Catanzaro, "Image Inpainting for Irregular Holes Using Partial Convolutions," Proceedings of the European Conference on Computer Vision, pp. 85-100, 2018.
14 J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, T. Huang, "Free-form Image Inpainting with Gated Convolution," Proceedings of IEEE/CVF International Conference on Computer Vision, pp. 4470-4479, 2019.
15 S. Iizuka, E. Simo-Serra, H. Ishikawa, "Globally and Locally Consistent Image Completion," ACM Transactions on Graphics, Vol. 36, No. 4, pp. 1-14, 2017.