ETRI Journal

  • 제37권6호
  • /
  • Pages.1199-1210
  • /
  • 2015
  • /
  • 1225-6463(pISSN)
  • /
  • 2233-7326(eISSN)
한국전자통신연구원 (Electronics and Telecommunications Research Institute)
권호 표지
DOI QR코드

DOI QR Code

Improved Two-Phase Framework for Facial Emotion Recognition

  • Yoon, Hyunjin (IT Convergence Technology Research Laboratory, ETRI) ;
  • Park, Sangwook (IT Convergence Technology Research Laboratory, ETRI) ;
  • Lee, Yongkwi (IT Convergence Technology Research Laboratory, ETRI) ;
  • Han, Mikyong (IT Convergence Technology Research Laboratory, ETRI) ;
  • Jang, Jong-Hyun (IT Convergence Technology Research Laboratory, ETRI)
  • 투고 : 2014.04.29
  • 심사 : 2015.11.11
  • 발행 : 2015.12.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Automatic emotion recognition based on facial cues, such as facial action units (AUs), has received huge attention in the last decade due to its wide variety of applications. Current computer-based automated two-phase facial emotion recognition procedures first detect AUs from input images and then infer target emotions from the detected AUs. However, more robust AU detection and AU-to-emotion mapping methods are required to deal with the error accumulation problem inherent in the multiphase scheme. Motivated by our key observation that a single AU detector does not perform equally well for all AUs, we propose a novel two-phase facial emotion recognition framework, where the presence of AUs is detected by group decisions of multiple AU detectors and a target emotion is inferred from the combined AU detection decisions. Our emotion recognition framework consists of three major components - multiple AU detection, AU detection fusion, and AU-to-emotion mapping. The experimental results on two real-world face databases demonstrate an improved performance over the previous two-phase method using a single AU detector in terms of both AU detection accuracy and correct emotion recognition rate.

키워드

참고문헌

  1. P. Ekman and W. Friesen, "The Facial Action Coding System: A Technique for the Measurement of Facial Movement," Palo Alto, CA, USA: Consulting Psychologists Press, 1978.
  2. W. Yun et al., "Hybrid Facial Representations for Emotion Recognition," ETRI J., vol. 35, no. 6, Dec. 2013, pp. 1021-1028. https://doi.org/10.4218/etrij.13.2013.0054
  3. Y. Li et al., "Data-Free Prior Model for Facial Action Unit Recognition," IEEE Trans. Affective Comput., vol. 4, no. 2, Apr. 2013, pp. 127-141. https://doi.org/10.1109/T-AFFC.2013.5
  4. G. Donato et al., "Classifying Facial Actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, 1999, pp. 974-989. https://doi.org/10.1109/34.799905
  5. Y. Zhu et al., "Dynamic Cascades with Bidirectional Bootstrapping for Action Unit Detection in Spontaneous Facial Behavior," IEEE Trans. Affective Comput., vol. 2, no. 2, Apr.-June 2011, pp. 79-91. https://doi.org/10.1109/T-AFFC.2011.10
  6. M.F. Valstar and M. Pantic, "Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics," IEEE Int. Conf. Human-Comput. Interaction, Rio de Janeiro, Brazil, Oct. 20, 2007, pp. 118-127.
  7. B. Jiang, M.F. Valstar, and M. Pantic, "Action Unit Detection Using Sparse Appearance Descriptors in Space-Time Video Volumes," IEEE Int. Conf. Automat. Face Gesture Recogn., Santa Barbara, CA, USA, Mar. 21-25, 2011, pp. 314-321.
  8. Z. Zeng et al., "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, Jan. 2009, pp. 39-58. https://doi.org/10.1109/TPAMI.2008.52
  9. I. Cohen et al., "Facial Expression Recognition from Video Sequences: Temporal and Static Modeling," Comput. Vis. Image Understanding, vol. 91, no. 1-2, July 2003, pp. 160-187. https://doi.org/10.1016/S1077-3142(03)00081-X
  10. M.S. Bartlett et al., "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., San Diego, CA, USA, vol. 2, June 20-25, 2005, pp. 568-573.
  11. K.-Y. Chang, T.-L. Liu, and S.-H. Lai, "Learning Partially-Observed Hidden Conditional Random Fields for Facial Expression Recognition," IEEE Conf. Comput. Soc. Vis. Pattern Recogn., Miami, FL, USA, June 20-25, 2009, pp. 533-540.
  12. Y. Zhang and Q. Ji, "Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, May 2005, pp. 699-714. https://doi.org/10.1109/TPAMI.2005.93
  13. P. Lucey et al., "The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit and Emotion-Specified Expression," IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. Workshop, San Francisco, CA, USA, June 13-18, 2010, pp. 94-101.
  14. M. Pantic and L.J.M. Rothkrantz, "An Expert System for Multiple Emotional Classification of Facial Expressions," IEEE Int. Conf. Tools Artif. Intell., Chicago, IL, USA, 1999, pp. 113-120.
  15. M.F. Valstar and M. Pantic, "Biologically vs. Logic Inspired Encoding of Facial Actions and Emotions in Video," IEEE Int. Conf. Multimedia Expo, Toronto, Canada, 2006, pp. 325-328.
  16. S. Velusamy et al., "A Method to Infer Emotions from Facial Action Units," IEEE Int. Conf. Acoust., Speech Signal Process., Prague, Czech Republic, May 22-27, 2011, pp. 2028-2031.
  17. Q. Ji, ISL Facial Expression Databases, Intelligent Systems Lab, Rensselear Polytechnic Institute. Accessed Feb. 26, 2014. http://www.ecse.rpi.edu/-cvrl/database/database.html
  18. N. Littlestone and M.K. Warmuth, "The Weighted Majority Algorithm," Inf. Comput., vol. 108, no. 2, Feb. 1994, pp. 212-261. https://doi.org/10.1006/inco.1994.1009
  19. Y.S. Huang and C.Y. Suen, "The Behavior-Knowledge Space Method for Combination of Multiple Classifiers," IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., New York, USA, June 15-17, 1993, pp. 347-352.
  20. D. Heckerman, "A Tutorial on Learning with Bayesian Networks," in Learning in Graphical Models, Cambridge, MA, USA: MIT Press, 1999, pp. 301-354.
  21. M.F. Valstar and M. Pantic, "Induced Disgust, Happiness and Surprise: An Addition to the MMI Facial Expression Database," Int. Conf. Language Resources Evaluation, Istanbul, Turkey, May 21-27, 2010, pp. 65-70.

피인용 문헌

  1. Adaptive Speech Streaming Based on Packet Loss Prediction Using Support Vector Machine for Software-Based Multipoint Control Unit over IP Networks vol.38, pp.6, 2015, https://doi.org/10.4218/etrij.16.2716.0013