Browse > Article
http://dx.doi.org/10.7840/KICS.2012.37A.4.239

Multi-view Generation using High Resolution Stereoscopic Cameras and a Low Resolution Time-of-Flight Camera  

Lee, Cheon (광주과학기술원 정보통신공학과)
Song, Hyok (전자부품연구원 멀티미디어IP)
Choi, Byeong-Ho (전자부품연구원 멀티미디어IP)
Ho, Yo-Sung (광주과학기술원 정보통신공학과)
Publication Information
The Journal of Korean Institute of Communications and Information Sciences / v.37, no.4A, 2012 , pp. 239-249 More about this Journal
Abstract
Recently, the virtual view generation method using depth data is employed to support the advanced stereoscopic and auto-stereoscopic displays. Although depth data is invisible to user at 3D video rendering, its accuracy is very important since it determines the quality of generated virtual view image. Many works are related to such depth enhancement exploiting a time-of-flight (TOF) camera. In this paper, we propose a fast 3D scene capturing system using one TOF camera at center and two high-resolution cameras at both sides. Since we need two depth data for both color cameras, we obtain two views' depth data from the center using the 3D warping technique. Holes in warped depth maps are filled by referring to the surrounded background depth values. In order to reduce mismatches of object boundaries between the depth and color images, we used the joint bilateral filter on the warped depth data. Finally, using two color images and depth maps, we generated 10 additional intermediate images. To realize fast capturing system, we implemented the proposed system using multi-threading technique. Experimental results show that the proposed capturing system captured two viewpoints' color and depth videos in real-time and generated 10 additional views at 7 fps.
Keywords
병합형 스테레오 카메라;3차원 카메라;3차원 비디오 시스템;깊이영상 생성;3차원 영상 보정;다시점 영상 생성;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. Zhu, L. Wang, R. Yang, and J. Davis, "Fusion of time-of-flight depth and stereo for high accuracy depth maps," IEEE CVPR, 2008.
2 E. K. Lee and Y. S. Ho, "Generation of Multi-view Video using a Fusion Camera System for 3D Displays," IEEE Transactions on Consumer Electronics, vol. 56, pp. 2797-2805, Nov. 2010.   DOI   ScienceOn
3 R. Y. Tsai, "Versatile Camera Calibration Technique for High-accuracy 3D Machine Vision Metrology using Off-The-Shelf TV Cameras and Lenses," IEEE Journal of Robotics and Automation, vol. RA-3, pp. 323-344, Aug. 1987.
4 Y. S. Kang, C. Lee, and Y. S. Ho, "An Efficient Rectification Algorithm for Multi-view Images in Parallel Camera Array," Proceedings of 3DTV-CON, pp. 61-64, 2008.
5 A. Telea, "An Image Inpainting Technique based on The Fast Marching Method," Journal Graphics Tools, vol. 9, pp. 25-36, May 2004.
6 C. Fehn, "Depth-Image-Based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV," in Stereoscopic Displays and Virtual Reality Systems XI, pp. 93-104, Jan. 2004.
7 ISO/IEC JTC1/SC29/WG11, "Study Text of ISO/IEC 14496-10:200x/FPDAM 1 Constrained Baseline Profile and supplementary enhancement information," in MPEG output document N10540, April 2009.
8 ISO/IEC JTC1/SC29/WG11, "Draft Call for Proposals on 3D Video Coding Technology," in MPEG output document N11679, Oct. 2010.
9 D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-frame Stereo Correspondence Algorithms," IJCV, vol. 47, pp. 7-42, April 2002.   DOI   ScienceOn
10 D. Scharstein and R. Szeliski, "High-accuracy Stereo Depth Maps using Structured Light," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. I/195-I/202, June 2003.
11 Y. M. Kim, D. Chan, C. Theobalt, and S. Thrun, "Design and calibration of a multi-view TOF sensor fusion system," IEEE CVPR, 2008.
12 R. Kaufmann, M. Lehmann, M. Schweizer, M. Richter, P. Metzler, G. Lang, T. Oggier, N. Blanc, P. Seitz, G. Gruener, and U. Zbinden, "A time-of-flight line sensor - Development and application," Proceedings of SPIE - The International Society for Optical Engineering, pp. 192-199, 2004.
13 M. Lindner, A. Kolb, and K. Hartmann, "Data-fusion of PMD-based distance-information and high-resolution RGB-images," International Symposium on Signals, Circuits and Systems, pp. 121-124, 2007.
14 B. Huhle, S. Fleck, and A. Schilling, "Integrating 3D Time-of-Flight Camera Data and High Resolution Images for 3DTV Applications," Proceedings of 3DTV-CON, 2007.
15 U. Hahne and M. Alexa, "Depth Imaging by Combining Time-of-Flight and On-demand Stereo," Lecture Notes in Computer Science, vol. 5742 , ed, pp. 70-83, 2009.
16 A. Smolic, K. Muller, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, "Intermediate View Interpolation based on Multiview Video plus Depth for Advanced 3D Video Systems," in IEEE ICIP, pp. 2448-2451, Oct. 2008.
17 C. Wheatstone, "On Some Remarkable, and Hitherto Unobserved, Phenomena of Binocular Vision," Philos. Trans. R. Soc., vol. 54, pp. 196-199, 1838.
18 Yo-Sung Ho, "Realistic 3D Content Producing using Multi-view and Depth Cameras," The Magazine of the IEEK, vol. 38, pp. 136-141, 2011.