Browse > Article
http://dx.doi.org/10.7746/jkros.2013.8.4.266

Robust Real-time Tracking of Facial Features with Application to Emotion Recognition  

Ahn, Byungtae (Robotics Program, KAIST)
Kim, Eung-Hee (School of Dentistry, Seoul National University)
Sohn, Jin-Hun (Department of Psychology, Chungnam National University)
Kweon, In So (Department of Electrical Engineering, KAIST)
Publication Information
The Journal of Korea Robotics Society / v.8, no.4, 2013 , pp. 266-272 More about this Journal
Abstract
Facial feature extraction and tracking are essential steps in human-robot-interaction (HRI) field such as face recognition, gaze estimation, and emotion recognition. Active shape model (ASM) is one of the successful generative models that extract the facial features. However, applying only ASM is not adequate for modeling a face in actual applications, because positions of facial features are unstably extracted due to limitation of the number of iterations in the ASM fitting algorithm. The unaccurate positions of facial features decrease the performance of the emotion recognition. In this paper, we propose real-time facial feature extraction and tracking framework using ASM and LK optical flow for emotion recognition. LK optical flow is desirable to estimate time-varying geometric parameters in sequential face images. In addition, we introduce a straightforward method to avoid tracking failure caused by partial occlusions that can be a serious problem for tracking based algorithm. Emotion recognition experiments with k-NN and SVM classifier shows over 95% classification accuracy for three emotions: "joy", "anger", and "disgust".
Keywords
facial feature; active shape model; optical flow; emotion recognition;
Citations & Related Records
연도 인용수 순위
  • Reference
1 T. F. Cootes, C. J. Taylor, D. H. Cooper, J. Graham, "Active Shape Models-Their Training and Application", Computer Vision Image Understand, Vol. 61, No. 9, pp. 38-59, 1995.   DOI   ScienceOn
2 T. F. Cootes, G. Edwards, C. Taylor. "Active appearance models", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, pp. 681-685, 2001.   DOI   ScienceOn
3 M. Zhou, L. Liang, J. Sun, Y. Wang, "AAM based Face Tracking with Temporal Matching and Face Segmentation", IEEE Conference on Computer Vision and Pattern Recognition, pp. 701-708, 2010.
4 J. K. Kearney, W. B. Thompson, "Optical Flow Estimation: An Error Analysis of Gradient-based Methods with Local Optimization", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No. 2, pp. 229-244, 1987.
5 M. Stephen, N. Fred, "Locating facial features with an extended active shape model", Proceedings of the 10th European Conference on Computer Vision. Vol. 4, pp. 504-513, 2008.
6 M. Kass, A. Witkin, D. Terzopoulos, "Snakes: Active Contour Models", International Journal of Computer Vision, Vol. 1, No. 4, pp. 321-331, 1988.   DOI   ScienceOn
7 R. Gross, I. Matthews, S. Baker, "Active Appearance Models with Occlusion", Image and Vision Computing, Vol. 24, No. 6, pp. 593-604, 2006.   DOI   ScienceOn
8 Antonio G., Sorci M., Bierlaire M., Thiran J., "Discrete choice models for static facial expression recognition", Proceedings of the 8th International Conference on Advanced Concepts for Intelligent Vision Systems, Vol. 4179, pp. 710-721, 2006.
9 Zhang Y., Ji Q., "Facial expression understanding in image sequences using dynamic and active visual information fusion", Proceedings of the 9th International Conference on Computer Vision, Vol. 2, pp. 1297-1304, 2003.
10 Bimler D., Paramei G., "Facial-expression affective attributes and their configural correlates: components and categories", Spanish Journal of Psychology, Vol. 9, pp. 19-31, 2006.   DOI
11 M. Pantic and L. J. M. Rothkrantz, "Case-based reasoning for user-profiled recognition of emotions from face images," Proceedings of the International Conference on Multimedia, pp. 391-394, 2004.
12 M. Valstar, M. Pantic, Z. Ambadar and J. F. Cohn, "Spontaneous versus posed facial behavior: Automatic analysis of brow actions", Proceedings of the International Conference on Multimodal Interfaces, pp. 162-170, 2006.