Browse > Article
http://dx.doi.org/10.5573/ieie.2015.52.7.093

Eye Contact System Using Depth Fusion for Immersive Videoconferencing  

Jang, Woo-Seok (Gwangju Institute of Science and Technology, School of Information and Communications)
Lee, Mi Suk (Electronics and Telecommunications Research Institute)
Ho, Yo-Sung (Gwangju Institute of Science and Technology, School of Information and Communications)
Publication Information
Journal of the Institute of Electronics and Information Engineers / v.52, no.7, 2015 , pp. 93-99 More about this Journal
Abstract
In this paper, we propose a gaze correction method for realistic video teleconferencing. Typically, cameras used in teleconferencing are installed at the side of the display monitor, but not in the center of the monitor. This system makes it too difficult for users to contact each eyes. Therefore, eys contact is the most important in the immersive videoconferencing. In the proposed method, we use the stereo camera and the depth camera to correct the eye contact. The depth camera is the kinect camera, which is the relatively cheap price, and estimate the depth information efficiently. However, the kinect camera has some inherent disadvantages. Therefore, we fuse the kinect camera with stereo camera to compensate the disadvantages of the kinect camera. Consecutively, for the gaze-corrected image, view synthesis is performed by 3D warping according to the depth information. Experimental results verify that the proposed system is effective in generating natural gaze-corrected images.
Keywords
Depth camera; eye contact; gaze correction; stereo matching; videonferencing;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. B. Lee, I. Y. Shin, and Y. S. Ho, "Gaze-corrected view generation using stereo camera system for immersive videoconferencing," IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1033-1040, 2011.   DOI   ScienceOn
2 C. Kuster, T. Popa, J. C. Bazin et al., "Gaze correction for home video conferencing," ACM Transaction on Graphics, vol. 31, no. 6, pp. 1-6, 2012.
3 J. Zhu, R. Yang, and X. Xiang, "Eye contact in video conference via fusion of time-of-flight depth sensor and stereo," 3D Research, vol. 2, no. 3, pp. 1-10, 2011.
4 Video and Requirement Group, "Call for Proposals on 3D Video Coding Technology " N12036, ISO/IEC JTC1/SC29/WG11, 2011.
5 L. Zhang, and T. Wa James, "Stereoscopic image generation based on depth images for 3D TV," IEEE Transactions on Broadcasting, vol. 51, no. 2, pp. 191-199, 2005.   DOI   ScienceOn
6 C. Fehn, and R. S. Pastoor, "Interactive 3-DTV-Concepts and Key Technologies," Proceedings of the IEEE, vol. 94, no. 3, pp. 524-538, 2006.   DOI
7 E. K. Lee, and Y. S. Ho, "Generation of multi-view video using a fusion camera system for 3D displays," IEEE Transactions on Consumer Electronics, vol. 56, no. 4, pp. 2797-2805, 2010.   DOI   ScienceOn
8 W. S. Jang, and Y. S. Ho, "Efficient disparity map estimation using occlusion handling for various 3D multimedia applications," IEEE Transactions on Consumer Electronics, vol. 57, no. 4, pp. 1937-1943, 2011.   DOI   ScienceOn
9 D. Scharstein, R. Szeliski, and R. Zabih, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms." pp. 131-140.
10 S. Y. Kim, S. B. Lee, and Y. S. Ho, "Three-dimensional natural video system based on layered representation of depth maps," IEEE Transactions on Consumer Electronics, vol. 52, no. 3, pp. 1035-1042, 2006.   DOI   ScienceOn
11 L. Xia, C. C. Chen, and J. K. Aggarwal, "Human detection using depth information by Kinect." pp. 15-22.
12 Z. Zhang, "A flexible new technique for camera calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, 2000.   DOI   ScienceOn
13 Y. S. Kang, and Y. S. Ho, "An efficient image rectification method for parallel multi-camera arrangement," IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1041-1048, 2011.   DOI   ScienceOn
14 W. S. Jang, and Y. S. Ho, "Efficient depth map generation with occlusion handling for various camera arrays," Signal, Image and Video Processing, vol. 8, no. 2, pp. 287-297, 2014/02/01, 2014.   DOI
15 Y. S. Kang, and Y. S. Ho, "High-quality multi-view depth generation using multiple color and depth cameras." pp. 1405-1410.
16 J. Kopf, M. F. Cohen, D. Lischinski et al., "Joint bilateral upsampling," ACM Trans. Graph., vol. 26, no. 3, pp. 96, 2007.   DOI   ScienceOn
17 U. Fecker, M. Barkowsky, and A. Kaup, "Improving the Prediction Efficiency for Multi-View Video Coding Using Histogram Matching." pp. 2-17.
18 R. Hartley, and A. Zisserman, Multiple View Geometry in Computer Vision, Second ed.: Cambridge University Press, 2004.