Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Fast Generation of Intermediate View Image Using GPGPU-Based Disparity Increment Method

GPGPU 기반의 변위증분 방법을 이용한 중간시점 고속 생성

  • Koo, Ja-Myung (Department of Electronic Materials Engineering, Kwangwoon University) ;
  • Seo, Young-Ho (Department of Liberal Arts, Kwangwoon University) ;
  • Kim, Dong-Wook (Department of Electronic Materials Engineering, Kwangwoon University)
  • Received : 2013.01.14
  • Accepted : 2013.08.05
  • Published : 2013.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Free-view, auto-stereoscopic video service is a next generation broadcasting system which offers a three-dimensional video, images of the various point are needed. This paper proposes a method that parallelizes the algorithm for arbitrary intermediate view-point image fast generation and make it faster using General Propose Graphic Processing Unit(GPGPU) with help of the Compute Unified Device Architecture(CUDA). It uses a parallelized stereo-matching method between the leftmost and the rightmost depth images to obtain disparity information and It use data calculated disparity increment per depth value. The disparity increment is used to find the location in the intermediate view-point image for each depth in the given images. Then, It is eliminate to disocclusions complement each other and remaining holes are filled image using hole-filling method and to get the final intermediate view-point image. The proposed method was implemented and applied to several test sequences. The results revealed that the quality of the generated intermediate view-point image corresponds to 30.47dB of PSNR in average and it takes about 38 frames per second to generate a Full HD intermediate view-point image.

자유시점 또는 오토스테레오스코픽 비디오 서비스는 3차원 영상을 제공하는 차세대 방송 시스템으로, 여러 시점의 영상들이 필요하다. 본 논문에서는 가상 시점 영상을 고속 생성하기 위해 알고리즘 병렬 구조를 최적화하고, Compute Unified Device Architecture(CUDA)를 이용한 General Propose Graphic Processing Unit(GPGPU) 기반의 중간시점 영상 고속 생성을 위한 최적화 기법을 제안한다. 제안한 방법은 좌/우 깊이영상을 병렬화시킨 스테레오 정합알고리즘을 이용하여 변위정보를 얻은 후, 깊이 당 변위증분을 계산하여 사용한다. 계산된 변위증분을 사용하여 해당 각 화소들의 깊이 값을 이용하여 좌/우 영상들을 원하는 위치의 중간시점으로 영상을 이동시킨다. 그 다음, 비폐색영역들을 서로 상호 보완하여 없앤 다음에 남은 홀들은 홀 필링으로 없애 최종 중간시점 영상을 생성한다. 제안한 방법을 구현하여 여러 실험 영상에 적용한 결과, 생성된 중간시점 깊이영상의 화질은 평균 PSNR 30.47dB이었으며, Full HD급 중간시점 영상을 초당 38 프레임 정도 생성하는 속도를 보였다.

Keywords

References

  1. Edited by O. Schreer, et al., 3D Video Communications, John Wiley & Sons Ltd., Atrium, England, 2005.
  2. ISO IWA3, Image safety reducing the incidence of undesirable biomedical effects caused by visual image sequences, 2005.
  3. 3D Consortium, "3DC Safety Guideline for Popularization of Human-friendly 3D," 2006.
  4. Li Yu, et al., "Depth based View Synthesis with Artifacts Removal for FTV," IEEE Intl. Conf. on Image and Graphics, pp. 506-510, 2011.
  5. Chia-Ming Cheng, et al., "Spatio-Temporally Consistent Novel View Synthesis Algorithm from Video-plus-Depth Sequences for Autostereoscopic Displays," IEEE Trans. on Broadcasting, Vol. 57, No. 2, pp. 523-532, June 2011. https://doi.org/10.1109/TBC.2011.2139090
  6. S. A. Benton and V. M. Bove, Jr., Holographic Imaging, John Wiley and Sons Inc., Hoboken NJ, 2008.
  7. Reichelt, et al., "Holographic 3-D Displays - Electroholography within the Grasp of Commercialization", A part of the Book named as Advances in Lasers and Electro Optics, INTECH, pp. 683-710, April 2010.
  8. J. L. Wilson, Microsoft kinect for Xbox 360, PC Mag. Com, Nov. 10, 2010.
  9. SR4000Data Sheet, http://www.mesa-imaging.ch/prodview 4k.php, MESA Imaging, Oct. 2010.
  10. A. Smolic, et al., "Intermediate View Interpolation based on Multiview Video plus Depth for Advanced 3D Video Systems," IEEE ICIP, pp.2448-2451, 2008.
  11. G. Cheung, et al., "On Dependent Bit Allocation for Multiview Image Coding with Depth-Image-Based Rendering", IEEE Trans. on Image Processing, Vol. 20, No. 11, pp. 3179-3194, Nov. 2011. https://doi.org/10.1109/TIP.2011.2158230
  12. S. Zinger, L. Do, and P. H. N. de With, "Free-viewpoint depth image based rendering," J. Vis. Commun. Image Representation, vol. 21, no. 5-6, pp. 533-541, 2010. https://doi.org/10.1016/j.jvcir.2010.01.004
  13. K.-J. Oh, S. Yea, and Y.-S. Ho, "Hole filling method using depth based inpainting for view synthesis in free viewpoint television and 3-D video," in Proc. Picture Coding Symp., Chicago, IL, May 2009, pp. 233-236.
  14. Jungsik Park, Ji-Youn Choi, In Ryu, and Jong-Il Park, "Universal View Synthesis Unit for Glassless 3DTV," IEEE Transactions on Consumer Electronics, 58(2): 706-711, May 2012 https://doi.org/10.1109/TCE.2012.6227480
  15. Luat Do, et al., "GPU-accelerated Real-time Free-viewpoint DIBR for 3DTV," IEEE Transactions on Consumer Electronics, 58(2): 633-640, May 2012 https://doi.org/10.1109/TCE.2012.6227470
  16. L. McMillan, Jr., "An image-based approach to threedimensional computer graphics," Ph.D. dissertation, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1997.
  17. A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, 2004. https://doi.org/10.1109/TIP.2004.833105
  18. I. Daribo, B. Pesquet "Depth-aided image inpainting for novel view synthesis," Multimedia Signal Processing (MMSP), IEEE International Workshop on pp 167-170, 2010
  19. J-M Koo, Y-H Seo, H-J Choi, J-S Yoo, and D-W Kim, "Intermediate Depth Image Generation using Disparity Increment of Stereo Depth Images", Journal of broadcast engineering, Vol. 17. Issue 2, pp. 363-373, March. 2012. https://doi.org/10.5909/JEB.2012.17.2.363
  20. http://wg11.sc29.org/svn/repos/MPEG-4/test/trunk/ 3Dview_synthesis/VSRS
  21. H. Shin, Y. Kim, H. Park, and J. Park, "Fast View Synthesis using GPU for 3D Display", In IEEE Trans. Consumer Electronics, vol. 54, no. 4, pp. 2068-2076, Nov. 2008. https://doi.org/10.1109/TCE.2008.4711274