Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Intermediate Depth Image Generation using Disparity Increment of Stereo Depth Images

스테레오 깊이영상의 변위증분을 이용한 중간시점 깊이영상 생성

  • Received : 2011.12.14
  • Accepted : 2012.02.21
  • Published : 2012.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a method to generate a depth image at an arbitrary intermediate view-point, which is targeting a video service for free-view, auto-stereoscopy, holography, etc. It assumes that the leftmost and the rightmost depth images are given and they both have been camera-calibrated and image-rectified. This method calculates and uses a disparity increment per depth value. In this paper, it is obtained by stereo matching for the given two depth image by considering more general cases. The disparity increment is used to find the location in the intermediate view-point depth image (IVPD) for each depth in the given images. Thus, this paper finds two IVPDs, from left image and from right image. Noises are removed and holes are filled in each IVPDs and the two results are combined to get the final IVPD. The proposed method was implemented and applied to several test sequences. The results revealed that the quality of the generated IVPD corresponds to 33.84dB of PSNR in average and it takes about 1 second to generate a HD IVPD. We evaluate that this image quality is quite good by considering the low correspondency among the left images, intermediate images, and the right images in the test sequences. If the execution speed is improved, the proposed method can be a very useful method to generate an IVPD at an arbitrary view-point, we believe.

본 논문에서는 자유시점 또는 오토스테레오스코픽 비디오나 홀로그래픽 비디오 서비스 등을 목표로 중간시점 깊이영상을 생성하는 방법을 제안한다. 이 방법에서 좌/우 깊이영상이 주어졌다고 가정하고, 이 깊이영상들은 카메라 보정과 영상보정이 수행되었다고 가정한다. 제안한 방법은 깊이 당 변위증분을 계산하여 사용하며, 이 변위증분은 일반적인 상황을 고려하여 좌/우 깊이영상을 스테레오 정합하여 계산한다. 계산된 변위증분을 사용하여 해당 깊이값의 위치를 원하는 중간시점 깊이영상에서 찾는다. 중간시점 깊이영상은 좌영상과 우영상으로부터 각각 생성하며, 잡음을 제거하고 홀 필링을 수행한 후 두 영상을 결합하여 최종 중간시점 깊이영상을 생성한다. 제안한 방법을 구현하여 여러 테스트 시퀀스에 적용한 결과, 생성된 중간시점 깊이영상의 화질은 평균 PSNR 33.84dB이었으며, HD급 중간시점 깊이영상을 초당 한 프레임 정도 생성하는 속도를 보였다. 이 화질은 테스트 시퀀스들의 비균질성을 고려할 때 상당히 좋은 화질이라 판단되며, 향후 수행속도를 더욱 향상시키면 제안한 방법은 중간시점 깊이영상을 생성하는 좋은 방법이 될 것으로 사료된다.

Keywords

References

  1. Li Yu, et al., "Depth based View Synthesis with Artifacts Removal for FTV," IEEE Intl. Conf. on Image and Graphics, pp. 506-510, 2011.
  2. Chia-Ming Cheng, et al., "Spatio-Temporally Consistent Novel View Synthesis Algorithm from Video-plus-Depth Sequences for Autostereoscopic Displays," IEEE Trans. on Broadcasting, Vol. 57, No. 2, pp. 523-532, June 2011. https://doi.org/10.1109/TBC.2011.2139090
  3. S. A. Benton and V. M. Bove, Holographic Imaging, John Wiley & Sons, Hoboken, NJ, 2008.
  4. S. Reichelt, et al., "Holographic 3-D Displays - Electro-holography within the Grasp of Commercialization", A part of the Book named as Advances in Lasers and Electro Optics, INTECH, pp. 683-710, April 2010.
  5. http://vision.middlebury.edu/stereo/.
  6. SR4000Data Sheet, http://www.mesa-imaging.ch/prodview4k.php, MESA Imaging, Oct. 2010.
  7. J. L. Wilson, Microsoft kinect for Xbox 360, PC Mag. Com, Nov. 10, 2010.
  8. A. Smolic, et al., "Intermediate View Interpolation based on Multiview Video plus Depth for Advanced 3D Video Systems," IEEE ICIP, pp.2448-2451, 2008.
  9. Y. Mori, et al., "View Generation with 3D Warping using Depth Information for FTV," Elsevier J.of Image Communications, Vol. 24, pp. 65-72, Oct. 2008.
  10. M. Zararim, et al., "A Joint Multivew plus Depth image Coding Scheme Based on 3D Warping," ACM Conf. on 3DVP, 2010.
  11. G. Cheung, et al., "On Dependent Bit Allocation for Multiview Image Coding with Depth-Image-Based Rendering", IEEE Trans. on Image Processing, Vol. 20, No. 11, pp. 3179-3194, Nov. 2011. https://doi.org/10.1109/TIP.2011.2158230
  12. E. K. Lee, et al., "High-Resolution Depth Map Generation by Applying Stereo Matching based on Initial Depth Information," IEEE 3DTV-CON, pp. 201-204, 2008.
  13. S. Y. Kim, et al., "Generation of ROI Enhanced Depth Maps using Stereoscopic Cameras and a Depth Camera, " IEEE Trans. on Broadcasting, Vol. 54, No. 4, pp. 732-739, Dec. 2008. https://doi.org/10.1109/TBC.2008.2002338
  14. Edited by Oliver Schreer, et al., 3D Video Communication, John Wiley and Sons, West Sussex, England, 2005.

Cited by

  1. Fast Generation of Intermediate View Image Using GPGPU-Based Disparity Increment Method vol.17, pp.8, 2013, https://doi.org/10.6109/jkiice.2013.17.8.1908
  2. Presentation Method Using Depth Information vol.18, pp.3, 2013, https://doi.org/10.5909/JBE.2013.18.3.409