Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Optimal Facial Emotion Feature Analysis Method based on ASM-LK Optical Flow

ASM-LK Optical Flow 기반 최적 얼굴정서 특징분석 기법

  • 고광은 (중앙대학교 전자전기공학부) ;
  • 박승민 (중앙대학교 전자전기공학부) ;
  • 박준형 (중앙대학교 전자전기공학부) ;
  • 심귀보 (중앙대학교 전자전기공학부)
  • Received : 2011.07.13
  • Accepted : 2011.08.10
  • Published : 2011.08.25
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an Active Shape Model (ASM) and Lucas-Kanade (LK) optical flow-based feature extraction and analysis method for analyzing the emotional features from facial images. Considering the facial emotion feature regions are described by Facial Action Coding System, we construct the feature-related shape models based on the combination of landmarks and extract the LK optical flow vectors at each landmarks based on the centre pixels of motion vector window. The facial emotion features are modelled by the combination of the optical flow vectors and the emotional states of facial image can be estimated by the probabilistic estimation technique, such as Bayesian classifier. Also, we extract the optimal emotional features that are considered the high correlation between feature points and emotional states by using common spatial pattern (CSP) analysis in order to improvise the operational efficiency and accuracy of emotional feature extraction process.

얼굴영상에서 나타나는 정서특징을 분석하기 위하여 본 논문에서는 Active Shape Model (ASM)과 Lucas-Kanade (LK) optical flow 기법을 기반으로 하는 특징검출 및 분석방법을 제안한다. Facial Action Coding System에 근거하여 묘사된 정서적 특징을 고려하여, 특징이 분포하는 영역에 위치한 다수의 landmark로 shape 모델을 구성하고 모델에서 각 Landmark를 중심으로 하는 움직임 벡터 윈도우 내부의 픽셀에 대한 LK 기법을 통해 optical flow 벡터를 추출한다. 추출된 움직임 벡터의 방향성 조합에 근거하여 얼굴정서특징을 shape 모델로 표현할 수 있으며, 베이지안 분류기라는 확률 기반 추론기법을 기반으로 정서적 상태에 대한 추정할 수 있다. 또한, 정서특징분석과정의 연산 효율성과 정확성 향상을 도모하기 위하여 common spatial pattern (CSP) 분석기법을 적용하여 정서상태 별로 상관성이 높은 특징만으로 구성된 최적정서특징을 추출한다.

Keywords

References

  1. X. Cufi, X. Munoz, J. Freixenet and J. Marti, "A review of image segmentation techniques integrating region and boundary information," Advances in Imaging and Electron Physics, vol. 120, pp. 1-39, 2003. https://doi.org/10.1016/S1076-5670(02)80033-7
  2. Y. J. Zhang, "A survey on evaluation methods for image segmentation," Pattern Recognition, vol. 29, no. 8, pp. 1335-4679, 1996.
  3. J. Liu and J. K. Udupa, "Oriented Active Shape Models," IEEE Transactions on Medical Imaging, vol. 28, no. 4, pp. 571-584, 2009. https://doi.org/10.1109/TMI.2008.2007820
  4. D. Terzopoulos and Kurt Fleischer, "Deformable models," The Visual Computer, vol. 4, no. 6, pp. 306-331, 1988. https://doi.org/10.1007/BF01908877
  5. M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active Contour Models, International Journal of Computer Vision, pp. 321-331, 1988.
  6. S. Sclaroff and L. Liu, "Deformable shape detection and description via model-based region grouping," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 5, pp. 475-489, 2001. https://doi.org/10.1109/34.922706
  7. T. F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham, "Active Shape Models-Their training and application," Computer Vision Image Understand, vol. 61, pp. 38-59, 1995. https://doi.org/10.1006/cviu.1995.1004
  8. J. K. Kearney, W.B. Thompson, "Optical flow estimation: an error analsysi of gradient-based methods with local optimization," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-9, no. 2, 1987. https://doi.org/10.1109/TPAMI.1987.4767897
  9. S. Y. Bin, Z.J. Ming, T.J. Hua, Z.G. Tao, "An improved facial feature localization method based on ASM," 7th International Conference on Computer-Aided Industrial Design and Conceptual Design, 2006.
  10. S. Kobayashi, S. Hashimoto, "Automated feature extraction of face image and its application," 4th IEEE International Workshop on Robot and Human Communication, pp. 164-169, 1995. https://doi.org/10.1109/ROMAN.1995.531954
  11. T. F Cootes, G. J. Edwards and C. J. Taylor, "Active Appearance Model," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, 2001. https://doi.org/10.1109/34.927467
  12. F. Arambula, J. A. Marquez Flores, M. A. Padilla Castaneda, "Use of simplex search in active shape models for improved boundary segmentation," Pattern Recognition Letters, vol. 31, Issue 9, pp. 806-817, 2010. https://doi.org/10.1016/j.patrec.2010.01.010
  13. T. F. Cootes, C. J. Taylor, Statistical Models of Appearance for Computer Vision, Technical Report, Volfson Image Analysis Unit, University of Manchester, UK, 2001.
  14. J. H. Shin, S. J Kim, S. K. Kang, S. W. Lee, J. K. Paik, B. Abidi, M. Abidi, "Optical flow-based real-time object tracking using non-prior training active feature model," Real-Time Imaging, vol. 11, no. 3, pp. 204-218, 2005. https://doi.org/10.1016/j.rti.2005.03.006
  15. P. Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press, Palo Alto, 1978.
  16. S. Milborrow, J. Morkel, F. Nicolls, "The MUCT Landmarked Face Database," Pattern Recognition Association of South Africa, 2010.
  17. M. J. Lyons, S. Akamatsu, M. Kamachi and J. Gyoba, "Coding facial expressions with Gabor wavelets," Proceedings of Third IEEE International Conference on Automatic Face and Gesture Recognition, IEEE Computer Society, pp. 200-205, 1998. https://doi.org/10.1109/AFGR.1998.670949
  18. P. Viola and M. J. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  19. 고광은, 심귀보, "FACS와 AAM을 이용한 Bayesian Network 기반 얼굴 표정 인식 시스템 개발," 한국지능시스템학회논문지, 제19권, 4호, pp. 562-567, 2009. https://doi.org/10.5391/JKIIS.2009.19.4.562
  20. Y. Wang, S. Gao, X. Gao, "Common spatial pattern method for channel selection in motor imagery based brain-computer interface," Proceeding of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 2005.
  21. 고광은, 심귀보, "ASM-LK optical flow 연동 기반 얼굴 정서 특징 분석," 한국지능시스템학회 2011년도 춘계학술대회 학술발표논문집, 제21권, 1호, pp. 175-178, 2011.