한국지능시스템학회논문지 (Journal of the Korean Institute of Intelligent Systems)

  • 제21권3호
  • /
  • Pages.395-400
  • /
  • 2011
  • /
  • 1976-9172(pISSN)
  • /
  • 2288-2324(eISSN)
한국지능시스템학회 (Korean Institute of Intelligent Systems)
권호 표지
DOI QR코드

DOI QR Code

Harmony Search 알고리즘 기반 HMM 구조 최적화에 의한 얼굴 정서 인식 시스템 개발

Development of Facial Emotion Recognition System Based on Optimization of HMM Structure by using Harmony Search Algorithm

  • 고광은 (중앙대학교 전자전기공학부) ;
  • 심귀보 (중앙대학교 전자전기공학부)
  • 투고 : 2011.02.21
  • 심사 : 2011.05.30
  • 발행 : 2011.06.25
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문에서는 얼굴 표정에서 나타나는 동적인 정서상태 변화를 고려한 얼굴 영상 기반 정서 인식 연구를 제안한다. 본 연구는 얼굴 영상 기반 정서적 특징 검출 및 분석 단계와 정서 상태 분류/인식 단계로 구분할 수 있다. 세부 연구의 구성 중 첫 번째는 Facial Action Units (FAUs)과 결합한 Active Shape Model (ASM)을 이용하여 정서 특징 영역 검출 및 분석기법의 제안이며, 두 번째는 시간에 따른 정서 상태의 동적 변화를 고려한 정확한 인식을 위하여 Hidden Markov Model(HMM) 형태의 Dynamic Bayesian Network를 사용한 정서 상태 분류 및 인식기법의 제안이다. 또한, 최적의 정서적 상태 분류를 위한 HMM의 파라미터 학습 시 Harmony Search (HS) 알고리즘을 이용한 휴리스틱 최적화 과정을 적용하였으며, 이를 통하여 동적 얼굴 영상 변화를 기반으로 하는 정서 상태 인식 시스템을 구성하고 그 성능의 향상을 도모하였다.

In this paper, we propose an study of the facial emotion recognition considering the dynamical variation of emotional state in facial image sequences. The proposed system consists of two main step: facial image based emotional feature extraction and emotional state classification/recognition. At first, we propose a method for extracting and analyzing the emotional feature region using a combination of Active Shape Model (ASM) and Facial Action Units (FAUs). And then, it is proposed that emotional state classification and recognition method based on Hidden Markov Model (HMM) type of dynamic Bayesian network. Also, we adopt a Harmony Search (HS) algorithm based heuristic optimization procedure in a parameter learning of HMM in order to classify the emotional state more accurately. By using all these methods, we construct the emotion recognition system based on variations of the dynamic facial image sequence and make an attempt at improvement of the recognition performance.

키워드

참고문헌

  1. M. Pantic, L.J.M. Rothkrantz, "Automatic analysis of facial expressions: the state of the art," IEEE Transactions on Pattern Anlysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, 2000. https://doi.org/10.1109/34.895976
  2. P. Ekman and W. Friesen, The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1, pp. 49-98.
  3. P. Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press, PaloAlto, 1978.
  4. C. Padgett and G. Cottrell, "Representing Face Images for Emotion Classification," Advances in Neural Information Processing Systems, vol. 9, 1997.
  5. M. J. Lyons, J. Budynek, and S. Akamatsu, "Automatic Classification of Single Facial Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, 1999. https://doi.org/10.1109/34.817413
  6. M. S. Bartlett, G. Littlewort, B. Braathen, T. J. Sejnowski, and J. R. Movellan, "An approach to automatic analysis of spontaneous facial expressions," Proc. 5th IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2002. https://doi.org/10.1109/AFGR.2002.1004180
  7. J.J. Bazzo and M.V. Lamar, "Recognizing Facial Actions Using Gabor Wavelets with Neutral Face Average Difference," Proc. 6th IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 505-510, 2004. https://doi.org/10.1109/AFGR.2004.1301583
  8. I. Kotsia and I. Pitas, "Facial Expression Recognition in Image Sequences using Geometric Deformation Features and Support Vector Machines," IEEE Transaction on Image Processing, vol. 16, no. 1, pp. 172-187, 2007. https://doi.org/10.1109/TIP.2006.884954
  9. Y.V. Venkatesh, Ashraf A. Kassim, O.V. Ramana Murthy, "A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA", Pattern Recognition Letters, vol. 30, Issue 12, pp. 1128-1137, 2009. https://doi.org/10.1016/j.patrec.2009.04.007
  10. K.E. Ko and K.B. Sim, "Emotion Recognition in Facial Image Sequences using a Combination of AAM with FACS and DBN," ICIRA 2010, Shanghai, China, Part I, LNAI 6424, pp. 702-712, 2010.
  11. 강효석, 백재호, 김은태, 박민용, "감정확률을 이용한 동적 얼굴표정의 퍼지모델링," 한국지능시스템학회 논문지, 제19권, 제1호, pp. 1-5, 2009
  12. 고광은, 심귀보, "FACS와 AAM을 이용한 Bayesian Network 기반 얼굴 표정 인식 시스템 개발," 한국지능시스템학회 논문지, 제19권, 제4호, pp. 562-567, 2009. https://doi.org/10.5391/JKIIS.2009.19.4.562
  13. Shi Yi-Bin, Zhang Jian-Ming, Tian Jian-Hua, Zhou Geng-Tao, "An improved facial feature localization method based on ASM," Computer-Aided Industrial design and Conceptual design, CAIDCD '06 7th international conference, 2006
  14. S. Kobayasho and S. Hashimoto, "Automated feature extraction of face image and its applications," International workshop on Robot and Human Communication, pp. 164-169.
  15. J. Liu and J. K. Udupa, "Oriented Active Shape Models", IEEE Transactions on medical Imaging, vol. 28, no. 4, pp. 571-584, 2009. https://doi.org/10.1109/TMI.2008.2007820
  16. T.F Cootes, G.J. Edwards and C.J. Taylor, "Active Appearance Model," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, 2001. https://doi.org/10.1109/34.927467
  17. I. Dryden, K. Mardia, Statistical Shape Analysis, Wiley, 1998.
  18. Y. Tong, W. Liao, Q. Ji, "Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1683-1699. https://doi.org/10.1109/TPAMI.2007.1094
  19. Y. Zhang, Q. Ji, "Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 699-714, 2005. https://doi.org/10.1109/TPAMI.2005.93
  20. Z. W. Geem, Music-Inspired Harmony Search Algorithm, Studies in Computational Intelligence, Springer, vol. 191, pp. 15-37.
  21. Z. W. Geem, K. B. Sim, "Parameter-setting-free harmony search algorithm," Applied Mathematics and Computation, vol. 217, Issue 8, pp. 3881-3889, 2010. https://doi.org/10.1016/j.amc.2010.09.049
  22. http://sites.google.com/a/hydroteq.com/www/
  23. K. J. Won, A. P-Bennett, and A. Krogh, "Training HMM structure with genetic algorithm for biological sequence analysis," Bioinformatics, vol. 20, no. 18, pp. 3613-3619, 2004. https://doi.org/10.1093/bioinformatics/bth454
  24. http://www.milbo.users.sonic.net/stasm/
  25. T. Mita, T. Kaneko, O. Hori, "Joint Haar-like Features for Face Detection", Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 2, pp. 1619-1626, 2005. https://doi.org/10.1109/ICCV.2005.129
  26. M.J. Lyons, J. Bydynek,and S. Akamatsu, "Automatic Classification of single Facial Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, 1999 https://doi.org/10.1109/34.817413
  27. S. Milborrow, J. Morkel, F. Nicolls, The MUCT Landmarked face database, Technical report, University of Cape Town, 2010