Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A High-Quality Occlusion Filling Method Using Image Inpainting

영상 인페인팅을 이용한 고품질의 가려짐 영역 보간 방법

  • Kim, Yong-Jin (Dept. of Electrical and Computer Engineering, Hanyang University) ;
  • Lee, Sang-Hwa (Dept. of Electrical Eng. and Computer Science, Seoul National University) ;
  • Park, Jong-Il (Dept. of Electrical and Computer Engineering, Hanyang University)
  • 김용진 (한양대학교 전자통신컴퓨터공학부) ;
  • 이상화 (서울대학교 전기공학부) ;
  • 박종일 (한양대학교 전자통신컴퓨터공학부)
  • Published : 2010.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method for filling out the occlusions in generating multi-view images from one source image and its ground-truth depth image. The method is based on image inpainting and layered interpolation. The source image is first divided into several layers using depth information. The occlusions are interpolated separately in every layered image using the image inpainting algorithm. Finally, the interpolated layered images are combined to obtain different viewpoint images. Interpolating occlusions with depth-correlated texture information that is contained to each layer makes it possible to obtain more detailed and accurate results than previous methods. The effectiveness of the proposed method is shown through experimental results.

본 논문에서는 한 장의 기준 영상과 그에 상응하는 참 깊이 맵을 이용하여 가상의 다중 시점 영상 생성 시 발생하는 가려짐 영역 보간 방법을 제안한다. 이 방법은 영상 인페인팅 기술과 각각의 깊이 정도에 따른 층별 보간 기술을 이용한다. 우선, 기준 영상을 깊이 정보에 따라 여러 개의 층으로 분할한다. 각각의 층에 대해 가려짐 영역 내의 화소들은 영상 인페인팅 기술을 이용하여 보간한다. 마지막 단계 에서 개별적으로 보간된 층 영상들은 하나로 합성되어 가상 시점의 영상을 이룬다. 영상을 깊이 정보에 따라 분할함으로써, 각 깊이 정도에 대한 텍스처의 연관성을 보존하며 보간할 수 있으므로 기존의 방법에 비하여 보다 정확하고 세밀한 가려짐 영역 보간이 가능하다. 본 논문에서는 여러 가지 실험 결과를 통하여 제안한 방법의 효율성을 입증하였다.

Keywords

References

  1. C. Fehn, "Depth-image-based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV," Proc. of SPIE, Stereoscopic Displays and Virtual Reality Systems XI, pp. 93-104, 2004.
  2. J. R. Ohm, "Stereo/Multi-view Video Encoding Using the MPEG Family of Standards," Proc. of Electronic Imaging, Invited Paper, 1999.
  3. A. Saxena, S. H. Chung, and A. Y. Ng, "3-D Depth Reconstruction from a Single Still Image," International Journal of Computer Vision (IJCV), 2007.
  4. A. Criminisi, P. Perez and K. Toyama, "Region Filling and Object Removal by Exemplar-based Image Inpainting," IEEE Trans. Image Processing, vol.13, no.9, pp. 1200-1212, 2004. https://doi.org/10.1109/TIP.2004.833105
  5. S. Battiatoa, S. Curtib, M. L. Casciac, M. Tortorac and E. Scordatoc, "Depth Map Generation by Image Classification," Proc. of SPIE Electronic Imaging, pp. 95-104, 2004.
  6. C. Fehn and P. Kauff, "Interactive Virtual View Video (IVVV): The Bridge between 3D-TV and Immersive TV," Proc. of SPIE Three-Dimensional TV, Video and Display, pp. 14-25, 2002.
  7. M. M. Oliveira, B. Bowen, R. McKenna and Y. S. Chang, "Fast Digital Image Inpainting," Proc. of the International Conference on Visualization Imaging and Image Processing, pp. 261-266, 2001.
  8. A. Efros and T. Leung, "Texture Synthesis by Non-parametric Sampling," Proc. of IEEE International Conference on Computer Vision , pp. 1033-1038, 1999.
  9. A. Efros and W. T. Freeman, "Image Quilting for Texture Synthesis and Transfer," Proc. of ACM SIGGRAPH, pp. 341-346, 2001.
  10. J.-F. Lalonde and A. Efors, "Using Color Compatibility for Assessing Image Realism," Proc. of IEEE International Conference on Computer Vision, pp. 1-8, 2007.
  11. I. Drori, D. Cohen-Or and H. Yeshurun, "Fragment-Based Image Completion," Proc. of ACM SIGGRAPH, Vol. 22, pp. 303-312, 2003. https://doi.org/10.1145/882262.882267
  12. S. Avidan and A. Shashua, "Novel View Synthesis by Cascading Trilinear Tensors," IEEE Trans. Visualization and Computer Graphics, vol.4 no.4, pp.293-306, 1998. https://doi.org/10.1109/2945.765324
  13. D. Scharstein, "View Synthesis Using Stereo Vision," Springer Press, 1999.
  14. J. Sun, L. Yuan, J. Jia and H. Y. Shum, "Image Completion with Structure Propagation," Proc. of ACM SIGGRAPH, Vol.24, pp. 861-868, 2005. https://doi.org/10.1145/1073204.1073274
  15. M. Bleyer, M. Gelautz, and C. Rother, C. Rhemann, "A stereo approach that handles the matting problem via image warping," Computer Vision and Pattern Recognition, 2009.
  16. K. Wegner and O. Stankiewicz, "Similarity measures for depth estimation," Proc. of 3DTV Conference: The True Vision," - Capture, Transmission and Display of 3D Video, 2009.
  17. K.-J. Oh, S. Yea, and Y.-S. Ho, "Hole-filling method using depth based in-painting for view synthesis in free viewpoint television (FTV) and 3D video," Proc. of Picture Coding Symposium (PCS), 2009.
  18. L. Zhang, W.J. Tam, "Stereoscopic image generation based on depth images for 3D TV," IEEE Trans. on Broadcasting, vol.51, no.2, pp. 191-199, 2005 https://doi.org/10.1109/TBC.2005.846190
  19. http://vision.middlebury.edu/stereo
  20. ITU-R Recommendation BT.500-10, "Methodology for the Subjective Assessment of the Quality of Television Pictures," 2002.

Cited by

  1. Study on the Methods of Enhancing the Quality of DIBR-based Multiview Intermediate Images using Depth Expansion and Mesh Construction vol.19, pp.1, 2015, https://doi.org/10.6109/jkiice.2015.19.1.127
  2. Hole Filling Algorithm for a Virtual-viewpoint Image by Using a Modified Exemplar Based In-painting vol.11, pp.4, 2016, https://doi.org/10.5370/JEET.2016.11.4.1003