The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지

Viewpoint interpolation of face images using an ellipsoid model

타원체 MODEL을 사용한 얼굴 영상의 시점합성에 관한 연구

  • 윤나리 (이화여자대학교 정보통신학과 영상처리연구실) ;
  • 이병욱 (이화여자대학교 정보통신학과)
  • Published : 2007.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

To establish eye contact in video teleconferencing, it is necessary to synthesize a front view image by viewpoint interpolation. We can find the viewing direction of a user, and interpolate an image seen from that viewpoint, which will result in a face image observed from the front. There are two categories of previous research: image based method and model based method. The former is simple to calculate, however, it shows limited performance for complex objects. And the latter is robust to noise while it is computationally expensive. We propose to approximate face images as ellipses and match them to build an ellipsoid and then synthesize a new image from a given virtual camera position. We show that it is simple and robust from various experiments.

영상 회의 시스템에서는 현실감을 높이기 위하여 대화 상대방과 시선을 맞추는 eye contact을 만들어 주는 것이 중요하다. 이를 위하여 영상 기반 방법과 모델 기반 방법 등의 시점합성기법이 연구되고 있다. 전자는 계산이 간단하나 부정확하고, 후자는 잡음에 강하고 정확한 영상을 만들어 내지만 시간이 많이 걸리는 단점이 있다. 본 논문에서는 얼굴 영상을 타원으로 근사화시키고 이를 3차원 공간에서 타원체로 모델링하여 임의의 가상 카메라의 위치에서 영상을 합성하는 방법을 제안하였다. 여러 가지 실험을 통하여 제안한 방법이 계산량이 적으면서도 성능이 우수함을 입증하였다.

Keywords

References

  1. D. W. Hanson, 'Eye typing off the shelf,' IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 159- 164, Washington D.C., U.S.A., Jul. 2004
  2. M. Z. Brown, D. Burschka, G. D .Hager, 'Advanced in computational stereo,' IEEE transaction on pattern analysis and machine intelligence, Vol. 25, No. 8, pp. 993-1008, Aug. 2003 https://doi.org/10.1109/TPAMI.2003.1217603
  3. P. Aschwanden and W. Guggenbuhl, 'Experimental Results from a comparative study on correlation-type registration algorithm,' Robust Computer Vision, Forstner and Ruwiedel, Eds., pp. 268-289, Wickmann, 1993
  4. V. Venkateswar and R. Chellappa, 'Hierarchical Stereo and Motion Correspondence Using Feature Groupings,' Int'l J. Computer Vision, Vol. 15, pp. 245-269, 1995 https://doi.org/10.1007/BF01451743
  5. S. Birchfield and C. Tomasi, ' Multiway cut for stereo and motion with slanted surfaces,' Proc. Int'l Conf. Computer Vision, Vol. 1, pp. 489-495, 1999
  6. S. Birchfield and C. Tomasi, 'Depth discontinuities by Pixel-to-Pixel Stereo,' Proceedings of the Sixth IEEE international Conference on Computer Vision, Mumbai, India, pp. 1073-1080, Jan. 1998
  7. C. Tomasi and R. Manduchi, 'Stereo without Search,' Technical Report STAN-CSTR-95-1543, Stanford Univ., 1995
  8. T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms. New York, McGraw-Hill, 1990
  9. 박창주, 고정환, 김은수, '스테레오 영상 보정 알고리즘에 기반한 새로운 중간시점 영상합성 기법,' 한국통신학회논문지, Vol. 29, No.. 5C, pp. 632-641, May. 2004
  10. R. Yang, Z. Zhang, 'Eye gaze correction with stereovision for video-teleconferencing,' IEEE transaction on Pattern Analysis and Machine Intelligence, Vol. 26, No. 7, pp. 956-960, Jul. 2004 https://doi.org/10.1109/TPAMI.2004.27
  11. H. Saito, M. Kimura, S. Yaguchi, N. Inamoto, 'View interpolation of multiple cameras based on projective geometry,' IPSJ Transactions on Computer Vision and Image Media, Vol.43, No. SIG 11, 2002
  12. J. Yin, and J. R. Cooperstock, 'Improving depth maps by nonlinear diffusion,' 12th International Conference on Computer Graphics, Visualization and Computer Vision, Feb. 2004
  13. 장세훈, 한충신, 배진우, 유지상, '깊이 정보를 이용한 실시간 다시점 스테레오 영상 합성,' 한국통신학회논문지, Vol. 30, No. 4C, pp. 239-246, Apr. 2005
  14. A. M. K. Siu and R. W. H. Lau, 'Image Registration for Image-Based Rendering,' IEEE Trans. on Image Processing, Vol. 14, No. 2, pp. 241-252, Feb. 2005 https://doi.org/10.1109/TIP.2004.840690
  15. Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen, 'Rapid modeling of animated faces from video,' Journal of Visualization and Computer Animation, Vol. 12, No. 4, pp. 227-240, 2001 https://doi.org/10.1002/vis.260
  16. M. Kass, A. Witkin, and D. Terzopoulos, 'Snake : Active Countour models,' Int'l J. Computer Vision, Vol. 1, No. 4, pp. 321-331, 1987
  17. M. H. Yang, D. Kriegman, N. Ahuja, 'Detecting Faces in Images: A Survey,' IEEE Trans. on Pattern Analysis and Machine Vision, Vol. 24, No. 1, pp. 34-58, Jan. 2004
  18. R. Tsai, 'A versatile Camera Calibration Technique for high accuracy 3D machine vision metrology using off the shelf TV cameras and lenses,' IEEE Journal of Robotics and Automation, Vol. 3, No. 4, pp. 323-344, 1987 https://doi.org/10.1109/JRA.1987.1087109
  19. S. A. MacKay, M. J. Potel, and J. M. Rubin, 'Graphics Methods for Tracking Three-Dimensional Heart Wall Motion,' Computers and Biomedical Research, Vol. 15, pp. 455-473, 1982 https://doi.org/10.1016/0010-4809(82)90027-1
  20. R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision, Cambridge University Press, 2000
  21. 박호식, 배철수, '휴먼 컴퓨터 인터페이스를 위한 실시간 시선 식별,' 한국통신학회논문지, Vol. 30, No. 3C, pp. 125-132, Mar. 2005
  22. 조상규, 황치규, 황재정, 'PC 카메라를 이용한 실시간 동공 검출,' 한국통신학회논문지, Vol. 29, No.. 8C, pp. 1184-1192, Aug. 2004
  23. R. Jain, R. Kasturi, B. G. Schunck, 'Machine Vision,' McGraw-Hill, 1995