Browse > Article
http://dx.doi.org/10.7840/KICS.2012.37C.4.277

Real-time Eye Contact System Using a Kinect Depth Camera for Realistic Telepresence  

Lee, Sang-Beom (광주과학기술원 정보통신공학과)
Ho, Yo-Sung (광주과학기술원 정보통신공학과)
Publication Information
The Journal of Korean Institute of Communications and Information Sciences / v.37, no.4C, 2012 , pp. 277-282 More about this Journal
Abstract
In this paper, we present a real-time eye contact system for realistic telepresence using a Kinect depth camera. In order to generate the eye contact image, we capture a pair of color and depth video. Then, the foreground single user is separated from the background. Since the raw depth data includes several types of noises, we perform a joint bilateral filtering method. We apply the discontinuity-adaptive depth filter to the filtered depth map to reduce the disocclusion area. From the color image and the preprocessed depth map, we construct a user mesh model at the virtual viewpoint. The entire system is implemented through GPU-based parallel programming for real-time processing. Experimental results have shown that the proposed eye contact system is efficient in realizing eye contact, providing the realistic telepresence.
Keywords
Eye contact system; gaze correction; depth camera; realistic telepresence; depth image-based rendering;
Citations & Related Records
연도 인용수 순위
  • Reference
1 D. Sharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," IEEE Workshop on Stereo and Multi-Baseline Vision, pp. 131-140, Dec. 2001.
2 C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation," SIGGRAPH'04, pp. 600-608, Aug. 2004.
3 D. Scharstein, and R. Szeliski, "High-accuracy stereo depth maps using structured light," Computer Vision and Pattern Recognition Workshops, vol. 1, pp. 195-202, June 2003.
4 S. Kim, S. Lee, and Y. Ho, "Three-dimensional natural video system based on layered representation of depth maps," IEEE Transactions on Consumer Electronics, vol. 52, no. 3, pp. 1035-1042, Aug. 2006.   DOI   ScienceOn
5 O. Schreer, N. Atzapadin, and I. Feldmann, "Multi-baseline disparity fusion for immersive videoconferencing," International Conference on Immersive Telecomm., pp. 27-29, May 2009.
6 E. Lee and Y. Ho, "Generation of multi-view video using a fusion camera system for 3D displays," IEEE Transactions on Consumer Electronics, vol. 56, no. 4, pp. 2797-2805, Nov. 2010.   DOI   ScienceOn
7 L. Xia, C. Chen, and J. K. Aggarwal, "Human detection using depth information by Kinect," Computer Vision and Pattern Recognition Workshops, pp. 15-22, June 2011.
8 Redert, M. O. Beeck, C. Fehn, W. IJsselsteijn, M. Pollefeys, L. Van Gool, E. Ofek, I. Sexton, P. Surman, "ATTEST: Advanced Three-dimensional Television System Techniques," International Symposium on 3D Data Processing, pp. 313-319, June 2002.
9 S. Lee, I. Shin, and Y. Ho, "Gaze-corrected view generation using stereo camera system for immersive videoconferencing," IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1033-1040, Aug. 2011.   DOI   ScienceOn
10 J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," SIGGRAPH'07, pp. 96-100, Aug. 2007.
11 S. Lee and Y. Ho, "Discontinuity-adaptive depth map filtering for 3D view generation," International Conference on Immersive Telecomm., pp. T8(1-6), 2009.