Journal of KIISE:Computing Practices and Letters (한국정보과학회논문지:컴퓨팅의 실제 및 레터)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

A Recognition Framework for Facial Expression by Expression HMM and Posterior Probability

표정 HMM과 사후 확률을 이용한 얼굴 표정 인식 프레임워크

  • 김진옥 (대구한의대학교 멀티미디어학부)
  • Published : 2005.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

I propose a framework for detecting, recognizing and classifying facial features based on learned expression patterns. The framework recognizes facial expressions by using PCA and expression HMM(EHMM) which is Hidden Markov Model (HMM) approach to represent the spatial information and the temporal dynamics of the time varying visual expression patterns. Because the low level spatial feature extraction is fused with the temporal analysis, a unified spatio-temporal approach of HMM to common detection, tracking and classification problems is effective. The proposed recognition framework is accomplished by applying posterior probability between current visual observations and previous visual evidences. Consequently, the framework shows accurate and robust results of recognition on as well simple expressions as basic 6 facial feature patterns. The method allows us to perform a set of important tasks such as facial-expression recognition, HCI and key-frame extraction.

본 연구에서는 학습한 표정 패턴을 기반으로 비디오에서 사람의 얼굴을 검출하고 표정을 분석하여 분류하는 프레임워크를 제안한다. 제안 프레임워크는 얼굴 표정을 인식하는데 있어 공간적 정보 외시간에 따라 변하는 표정의 패턴을 표현하기 위해 표정 특성을 공간적으로 분석한 PCA와 시공간적으로 분석한 Hidden Markov Model(HMM) 기반의 표정 HMM을 이용한다. 표정의 공간적 특징 추출은 시간적 분석 과정과 밀접하게 연관되어 있기 때문에 다양하게 변화하는 표정을 검출하여 추적하고 분류하는데 HMM의 시공간적 접근 방식을 적용하면 효과적이기 때문이다. 제안 인식 프레임워크는 현재의 시각적 관측치와 이전 시각적 결과간의 사후 확률 방법에 의해 완성된다. 결과적으로 제안 프레임워크는 대표적인 6개 표정뿐만 아니라 표정의 정도가 약한 프레임에 대해서도 정확하고 강건한 표정 인식 결과를 보인다. 제안 프레임 워크를 이용하면 표정 인식, HCI, 키프레임 추출과 같은 응용 분야 구현에 효과적이다

Keywords

References

  1. M. Rosenblum, Y. Yacoob and L. S. Davis, 'Human expression recognition from motion using a radial basis function network architecture,' IEEE Transactions on Neural networks, Vol. 7, No.5, pp. 1121-1138, 1996 https://doi.org/10.1109/72.536309
  2. B. Fasel, 'Multiscale Facial Expression Recognition using Convolutional Neural Networks,' In Proc. of the Third Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP'2002), 2002
  3. M. N. Daile, W. C. Cottrell, C. Padgett and R. Adlophs, 'EMPATH: A Neural Network that Categorizes Facial Expressions,' Journal of Cognitive Science, vol. 14, no. 8, pp. 1158-1173, 2002 https://doi.org/10.1162/089892902760807177
  4. M.J. Lyons, J. Budynek and S. Akamatsu, 'Automatic Classification of Single Facial Images,' IEEE Trans. Pattern Anal. Machine Intell., vol. 21, no. 12, pp. 1357-1362, 1999 https://doi.org/10.1109/34.817413
  5. M. Pantic and L. Rothrkantz, 'Expert System for Automatic Analysis of Facial Expression,' J. Image and Vision Computing, vol. 18, no. 11, pp. 881-905, 2000 https://doi.org/10.1016/S0262-8856(00)00034-2
  6. M. Schulze, K. Scheffeller and C. W. OmIin, 'Recognizing Facial Actions with Support Vector Machines,' In Proc. PRASA 2002, pp. 93-96, 2002
  7. Essa, I., and Pentland, A., 'Coding, Analysis, Interpretation, and Recognition of Facial Expressions,' IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.19, pp.757-763, 1997 https://doi.org/10.1109/34.598232
  8. T. Otsuka and J, Ohya, 'Spotting Segments Displaying Facial Expression from Image Sequences Using HMM,' In Proc. Inter. Conf. on Automatic Face and Gesture Recognition 98, pp. 442-477, 1998 https://doi.org/10.1109/AFGR.1998.670988
  9. J. J. Lien, H. Kanade, T. Kitamura, J. F. Cohn and C. C. Li, 'Detection, Tracking and Classification of Action Units in Facial Expression,' In Journal of Robotics and Autonomous Systems, pp. 432-329, 1998 https://doi.org/10.1016/S0921-8890(99)00103-7
  10. P. Ekman and W. V. Friesen, 'The Facial Action Coding System: A Technique for Measurement of Facial Movement,' Consulting Psychologists Press, San Francisco, CA, 1978
  11. S. Muller, S. Eickerler and G. Rigoll, 'Pseudo 3D HMMs for Image Sequence Recognition,' in IEEE Proc. Inter. Conf. on Image Processing. 1999, pp. 237-241, 1999
  12. J. Hoey, 'Hierarchical Unsupervised Learning of Facial Expression Categories,' IEEE Workshop on Detection and Recognition of Events in Video (EVENT'01), pp. 92-99, 2001 https://doi.org/10.1109/EVENT.2001.938872
  13. 이경아, '웨이블렛 계수와 Hidden Markov Model를 이용한 얼굴인식 기법', 한국 퍼지 및 지능시스템학회 03년 추계 학술대회 학술발표 논문집, 162-165쪽, 2003
  14. L. R. Rabiner, 'A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,' Pro. IEEE, vol 77, no. 2, pp. 257-286, 1989 https://doi.org/10.1109/5.18626
  15. C. Bregler, 'Learning and recognizing human dynamics in video sequences,' Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 568-574, 1998 https://doi.org/10.1109/CVPR.1997.609382
  16. Jin Ok Kim, Sung Jin Seo and Chin Hyun Chung, 'Real-Time Face Recognition by the PCA with Color Images,' Lecture Notes in Computer Science, LNCS 3036, pp. 687-690, June 2004
  17. 최명근, 정현숙, 신영숙, 이일병, '표정 변화에 따른 얼굴표정에서의 특징점 추적', 한국정보과학회 논문집 7권, 2호, 425-427쪽, 2000
  18. C. Choi, S. Kim, and W. Choi, 'Survivality Modeling for Quantitative Security Assessment in Ubiquitous Computing Systems,' Lecture Notes in Computer Science, Springer, Vol.3043, No.1, pp.207-214, May, 2004 https://doi.org/10.1007/b98048
  19. Viterbi, A. J. (1967). Error bounds for convolution codes and an asymptotically optimal decoding algorithm. IEEE trans. on Information Theory, 12:260-269 https://doi.org/10.1109/TIT.1967.1054010
  20. Y. Zhang, Q. Ji, 'Facial Expression Understanding in Image Sequences Using Dynamic and Active Visual Information Fusion,' IEEE Inter. Conf. on . Computer Vision (ICCV2003) , vol. 2, 2003
  21. C. E. Shannon, 'A mathematical theory of communication,' Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, 1948