The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지
DOI QR코드

DOI QR Code

Depth Video Post-processing for Immersive Teleconference

원격 영상회의 시스템을 위한 깊이 영상 후처리 기술

  • 이상범 (광주과학기술원 정보통신공학과) ;
  • 양승준 (한국전자통신연구원 방통융합미디어연구부) ;
  • 호요성 (광주과학기술원 정보통신공학과)
  • Received : 2012.02.11
  • Accepted : 2012.06.07
  • Published : 2012.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we present an immersive videoconferencing system that enables gaze correction between users in the internet protocol TV (IPTV) environment. The proposed system synthesizes the gaze corrected images using the depth estimation and the virtual view synthesis algorithms as one of the most important techniques of 3D video system. The conventional processes, however, causes several problems, especially temporal inconsistency of a depth video. This problem leads to flickering artifacts discomforting viewers. Therefore, in order to reduce the temporal inconsistency problem, we exploit the joint bilateral filter which is extended to the temporal domain. In addition, we apply an outlier reduction operation in the temporal domain. From experimental results, we have verified that the proposed system is sufficient to generate the natural gaze-corrected image and realize immersive videoconferencing.

본 논문에서는 IPTV 환경의 원격 영상회의에서 화자 간의 자연스러운 시선 맞춤(eye contact)을 위한 깊이영상의 후처리 필터링 기술을 제안한다. 제안하는 방법은 깊이탐색 기술과 영상합성 기술을 사용해서 화자의 정면시점 영상을 합성한다. 하지만, 깊이영상을 탐색하는 과정에서 객체의 경계 불일치, 시간적 상관도 저하 등의 문제가 발생하기 때문에 이를 해결하기 위해 시간축으로 확장된 결합형 양방향 필터(joint bilateral filter)를 제안한다. 실험 결과를 통해, 제안하는 깊이영상의 후처리 필터링 기술이 정면시점 합성영상의 화질을 향상시켰고, 원격의 화자와 시선 맞춤이 기능한 것을 확인했다.

Keywords

References

  1. P. Kauff and O. Schreer, "An immersive 3D video-conferencing system using shared virtual team user environments," Proc. of international conference on Collaborative virtual environments, pp. 105-112, Sep. 2002.
  2. S. Huang and J. Wang, "A low-cost desktop videoconferencing codec: an adaptive Motion-JPEG design," IEEE Transactions on Consumer Electronics, vol. 40, no. 4, pp. 944-950, Nov. 1994. https://doi.org/10.1109/30.338344
  3. S. M. Kuo, Y. C. Huang, and Z. Pan, "Acoustic noise and echo cancellation microphone system for videoconferencing," IEEE Transactions on Consumer Electronics, vol. 41, no. 4, pp. 1150-1158, Nov. 1995. https://doi.org/10.1109/30.477235
  4. H. Kwon, H. Han, S. Lee, W. Choi, and B. Kang, "New video enhancement preprocessor using the region-of-interest for the videoconferencing," IEEE Transactions on Consumer Electronics, vol. 56, no. 4, pp. 2644-2651, Nov. 2010. https://doi.org/10.1109/TCE.2010.5681152
  5. O. Schreer, N. Atzapadin, and I. Feldmann, "Multi-baseline disparity fusion for immersive videoconferencing," In Proceedings of International Conference on Immersive Telecommunications, pp. 27-29, May 2009.
  6. P. Kauff and O. Schreer, "An immersive 3D video-conferencing system using shared virtual team user environments," In Proceedings of International Conference on Collaborative Virtual Environments, pp. 105-112, Oct. 2002.
  7. S. Lee, I. Shin, and Y. Ho, "Gaze-corrected View Generation using Stereo Camera System for Immersive Videoconferencing," IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1033-1040, 2011. https://doi.org/10.1109/TCE.2011.6018852
  8. Q. Yang, L. Wang, and N. Ahuja, "A constant-space belief propagation algorithm for stereo matching," in Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1458-1465, 2010.
  9. P. Lai, D. Tian, and P. Lopez, "Depth map processing with iterative joint multilateral filtering," in Proceedings of Picture Coding Symposium, pp. 9-12, 2010.
  10. ISO/IEC JTCl/SC29/WG11, "Reference Software of Depth Estimation and View Synthesis for FTV/3DV," M15836, Oct. 2008.
  11. M. Sugawara, K. Mitani, M. Kanazawa, F. Okano, and Y. Nishida, "Future prospects of HDTV - technical trends toward 1080p," SMPTE Journal, vol. 115, pp. 10-15, 2006. https://doi.org/10.5594/J11496