Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Recognition of Hmm Facial Expressions using Optical Flow of Feature Regions

얼굴 특징영역상의 광류를 이용한 표정 인식

  • 이미애 (국립한밭대학교 BK21 사업단) ;
  • 박기수 (고신대학교 정보미디어학부)
  • Published : 2005.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

Facial expression recognition technology that has potentialities for applying various fields is appling on the man-machine interface development, human identification test, and restoration of facial expression by virtual model etc. Using sequential facial images, this study proposes a simpler method for detecting human facial expressions such as happiness, anger, surprise, and sadness. Moreover the proposed method can detect the facial expressions in the conditions of the sequential facial images which is not rigid motion. We identify the determinant face and elements of facial expressions and then estimates the feature regions of the elements by using information about color, size, and position. In the next step, the direction patterns of feature regions of each element are determined by using optical flows estimated gradient methods. Using the direction model proposed by this study, we match each direction patterns. The method identifies a facial expression based on the least minimum score of combination values between direction model and pattern matching for presenting each facial expression. In the experiments, this study verifies the validity of the Proposed methods.

표정인식 연구는 맨$\cdot$머신 인터페이스 개발, 개인 식별, 가상모델에 의한 표정복원 등 응용가치의 무한한 가능성과 함께 다양한 분야에서 연구되고 있다 본 논문에서는 인간의 기본정서 중 행복, 분노, 놀람, 슬픔에 대한 4가지 표정을 얼굴의 강체 움직임이 없는 얼굴동영상으로부터 간단히 표정인식 할 수 있는 방법을 제안한다. 먼저, 얼굴 및 표정을 결정하는 요소들과 각 요소의 특징영역들을 색상, 크기 그리고 위치정보를 이용하여 자동으로 검출한다. 다음으로 Gradient Method를 이용하여 추정한 광류 값으로 특징영역들에 대한 방향패턴을 결정한 후, 본 연구가 제안한 방향모델을 이용하여 방향패턴에 대한 매칭을 행한다. 각 정서를 대표하는 방향모델과의 패턴 매칭에서 그 조합 값이 최소를 나타내는 부분이 가장 유사한 정서임을 판단하고 표정인식을 행한다. 마지막으로 실험을 통하여 본 논문의 유효성을 확인한다.

Keywords

References

  1. P. Ekman, W. V. Friesen, Unmasking the Face, Prentice-Hall, Inc., Englewood Cliffs, 1975
  2. A. Mehrabian, J. A. Russell, An Approach to Environmental Psychology, MIT Press, 1974
  3. H. Kobayashi, F. Hara, 'A basic study of man-machine interactive communication by using facial expressions,' 9th Symposium on Human Interface, pp. 361-366, 1993
  4. O. Hasegawa, K. Sakaue and S. Hayamizu, 'A Human-Like software robot which interactively learns and manages visual information in real world,' IEICE, Vol. J82-D-II, No.10, pp. 1666-1674, 1999
  5. O. Yamaguchi, K. Fukui, 'Smartface-A robust face recognition system under varying facial pose and expression,' IEICE, Vol. J84-D-II, No.6, pp. 1045-1052, 2001
  6. K. Aizawa, H. Harashima and T. Saito, 'A model-based analysis synthesis image coding scheme,' IEICE, Vol.J72-D-I, No.3, pp. 200-207, 1989
  7. P. Vanger, R. Honlinger and H. Haken, 'Application of synergistic in decoding facial expressions of emotion,' Proceedings of IEEE International Workshop on Automatic Face and Gesture Recognizing, pp.44-49, 1995
  8. F. Hara, H. Kobayashi, 'Use of face Robot for Human-Computer communication,' IEEE Conference on SMC, pp.1515-1520, 1995 https://doi.org/10.1109/ICSMC.1995.537987
  9. C. Choi, H. Harashima and T. Takebe, 'Analysis of facial expressions using Three-Dimensional facial model,' IEICE, Vol. J74-D-II, No.6, pp. 766-777, 1991
  10. Y. Tian, T. Kanade and J. F. Cohn, 'Recognizing actions units for facial expression analysis,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No.2, pp. 97-115, 2001 https://doi.org/10.1109/34.908962
  11. M. Rosenblum, Y. Yacoob and L. S. Davis, 'Human expression recognition from motion using a radial basis function network architecture,' IEEE Transactions on Neural networks, Vol. 7, No.5, pp. 1121-1138, 1996 https://doi.org/10.1109/72.536309
  12. I. Essa, A. Pentland, 'Facial expression recognition using image motion,' Kluwer Academic Publishers, Computational Imaging and Vision Series, 1997
  13. P. Ekman, W. V. Friesen, Facial Action Coding System, Palo Alto, CA, Consulting Psychologi Press, 1978
  14. K. Mase, 'Recognition of facial expression from optical flow,' IEICE, Vol. E74, No.10, pp. 3474-3483, 1991
  15. B. K. P. Horn, B. B. Schunck, 'Determining optical flow,' Artificial Intelligence, Vol. 17, pp. 185-203, 1981 https://doi.org/10.1016/0004-3702(81)90024-2
  16. H. Ohashi, G. Xu, S. Tsuji, 'Recognition of facial expressions by neural network: A study on input data,' IEICE, PRU-91-4, pp. 23-29, 1991
  17. 이미애, 박기수, '컬러정보와 부분 템플릿을 이용한 얼굴영역, 요소 및 회전각 검출', 정보처리학회 논문지, 제10-B권, 제4호, pp. 465-472, 2003 https://doi.org/10.3745/KIPSTB.2003.10B.4.465
  18. J. K. Kearney, W. B. Thompson and L. Boley, 'Optical flow estimation: An arror analysis of Gradient-based methods with local optimization,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No.2, pp. 229-244, 1987
  19. FACS-Facial Action Coding System(Ekman and Friesen), http://www-2.cs.cmu.edu/~face/facs.htm
  20. N. Bratchell, 'Cluster analysis,' Chemometrics and Intelligent Laboratory systems, pp. 105-125, 1989