Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Region-Based Facial Expression Recognition in Still Images

  • Nagi, Gawed M. (Faculty of Computer Science and Information Technology, University Putra Malaysia (UPM)) ;
  • Rahmat, Rahmita O.K. (Faculty of Computer Science and Information Technology, University Putra Malaysia (UPM)) ;
  • Khalid, Fatimah (Faculty of Computer Science and Information Technology, University Putra Malaysia (UPM)) ;
  • Taufik, Muhamad (Faculty of Computer Science and Information Technology, University Putra Malaysia (UPM))
  • Received : 2012.10.24
  • Accepted : 2013.03.06
  • Published : 2013.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In Facial Expression Recognition Systems (FERS), only particular regions of the face are utilized for discrimination. The areas of the eyes, eyebrows, nose, and mouth are the most important features in any FERS. Applying facial features descriptors such as the local binary pattern (LBP) on such areas results in an effective and efficient FERS. In this paper, we propose an automatic facial expression recognition system. Unlike other systems, it detects and extracts the informative and discriminant regions of the face (i.e., eyes, nose, and mouth areas) using Haar-feature based cascade classifiers and these region-based features are stored into separate image files as a preprocessing step. Then, LBP is applied to these image files for facial texture representation and a feature-vector per subject is obtained by concatenating the resulting LBP histograms of the decomposed region-based features. The one-vs.-rest SVM, which is a popular multi-classification method, is employed with the Radial Basis Function (RBF) for facial expression classification. Experimental results show that this approach yields good performance for both frontal and near-frontal facial images in terms of accuracy and time complexity. Cohn-Kanade and JAFFE, which are benchmark facial expression datasets, are used to evaluate this approach.

Keywords

References

  1. Ekman P. and Friesen W. V., "Constant across cultures in the face and emotion," Journal of personality and social psychology, 1971, pp.124-129.
  2. Mehrabian, A. "Silent messages: Implicit communication of emotions and attributes (2nd edition)," Belmont, CA: Wadsworth Publishing Company, 1981.
  3. Hsu C.-W., Chang C.-C., Lin C.-J., "A Practical Guide to Support Vector Classification," Tech. Rep., Taipei, 2003.
  4. Shan C., Gong S., and McOwan P., "Facial expression recognition based on local binary patterns: A comprehensive study," Image and Vision Computing, 2009, pp.803-816.
  5. Bartlett M., GLittle wort C., Lainscsek C ., Fasel I., Frank M., and Movellan J., "Fully automatic facial action recognition in spontaneous behavior," Automatic Face and Gesture Recognition , 2006, pp.223-228.
  6. Giorgana G., "Facial Expression Recognition for Domestic Service Robots," Proceedings of SIMPAR 2010 Workshops, 2010, pp.251-262.
  7. Ryan A., Jeffery F. Cohn, and Adam Rossi, "Automated Facial Expression Recognition System," in IEEE 43rd International Carnahan Conference on Security Technology, Zurich, 2009, pp.172-177.
  8. Lu H., Huang Y., Chen Y. -W., and Yang D., "Real-time facial expression recognition based on pixel- pattern-based texture feature," Electronic Letters, 43 (17), 2007, pp.916-918. https://doi.org/10.1049/el:20070362
  9. Nomoto N. and Otsu N., "A New Scheme for Image Recognition using Higher-order Local Autocorrelation and Factor Analysis," MVA2005, 2005.
  10. Kurita T., Otsu N., and Sato T., "A face recognition method using higher order local autocorrelation and multivariate analysis," Proc. 11th IAPR International Conf. on Pattern Recognition, 1992, pp.213-216.
  11. Kurita T., and Hayamizu S., "Gesture Recognition using HLAC Features of PARCOR Images and HMM based Recognizer," IEEE Int. Conf. on Automatic Face and Gesture Recognition , Japan,1998.
  12. Lajevardi, S., Hussain, Z., "Automatic facial expression recognition: feature extraction and Selection," Signal, Image and Video Processing, 2011, pp.1-11.
  13. Kanade T., Cohn J. F., and Tian Y., "Comprehensive Database for Facial Expression Analysis," in Fourth IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000, pp.46-53.
  14. Lyons M. J., Akamatsu S., Kamachi M., and Gyoba J., "Coding Facial Expressions with Gabor Wavelets," in Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998, pp.200-205.
  15. Viola P., and Jones M., "Rapid Object Detection Using a Boosted Cascade of Simple Features," proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , Kauai, Hawaii, Vol.1, 2001, pp.511-518.
  16. Lienhart R., and Maydt J., "An extended set of haar-like features for rapid object detection," Proceedings of the IEEE International Conference o n Image Processing, Rochester, New York, Vol.1, 2002, p. 900-903.
  17. Li S.Z., and Zhang Z., "FloatBoost Learning and Statistical Face Detection", PAM I, 2004, pp.1112-1123.
  18. Mita T., Kaneko T., and Hori O., "Joint Haar-like features for face detection," Proceedings of the International Conference on Computer Vision, Washington, DC, USA, 2005, pp.1619-1626.
  19. Sri-Kaushik Pavani, David Delgado, and Alejandro F. Frangi, "Haar-like features with optimally weighted rectangles for rapid object detection," Pattern Recognition,2010, pp.160-172.
  20. "Face Detection using OpenCV", Available at: http://opencv.willowgarage.com/wiki/FaceDetection
  21. Lienhart R., Kuranov A., and Pisarevsky V., "Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection," Intel Corporation , Technical report, 2002, pp.297-304.
  22. Viola, P., & Jones, M. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137-154. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  23. Lefkovits, S. (2009). Assessments of building classifiers for face detection. Proceedings of International Conference on Recent Achievements in Mechatronics, Automation, Computer Sciences and Robotics MACRO (pp.175-186).
  24. Yin L., Wei X., Sun Y., Wang J., and Rosato M., " A 3D facial expression database for facial behavior research," In Proceedings of 7th International Conference on Automatic Face and Gesture Recognition, 2006, pp.211-216.
  25. Castrillon M.-Santana, Deniz-Suarez O., Anton-Canals L., and Lorenzo-Navarro J., "Face and facial feature detection evaluation: Performance Evaluation of Public Domain Haar Detectors for Face and Facial Feature Detection," in VISAPP, 2008.
  26. "Haar Cascades ", Available at: http://alereimondo.no-ip.org/OpenCV/34
  27. Ojala T., Pietikainen M., and Harwood D., "A comparative study of texture measures with classification based on featured distribution," Pattern Recognition,1996, pp.51-59.
  28. Huang D., Shan C., Ardabilian M., Wang Y., and Chen L., "Local binary patterns and its application to facial image analysis: a survey," IEEE Trans. on Systems, Man, and Cybernetics, Part C: Applications and Reviews 41 (4), 2011, pp.1-17. https://doi.org/10.1109/TSMCA.2010.2055152
  29. Ojala T., Pietikainen M., and Maenpaa T., "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, pp.971-987.
  30. Altun H. and Polat G., "New frameworks to boost feature selection algorithms in emotion detection for improved human-computer interaction," Brain Vision and Artificial Intelligent. Lecture Notes in Computer Science , 2007, pp.533-541.
  31. "The Spider": http://people.kyb.tuebingen.mpg.de/spider/
  32. Feng X., Hadid A., and Pietikainen M., "A coarse-to-fine classification scheme for facial expression recognition," in Proc. Int. Conf. Image Analysis and Recognition (ICIAR), 2004, pp.668-675.
  33. Liao S., Fan W., Chung A. C. S., and Yeung D. Y., "Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features," in Proc. IEEE Int. Conf. Image Processing (ICIP), 2006, pp.665-668.
  34. Zhao G.and Pietikainen M., "Experiments with facial expression recognition using spatiotemporal local binary patterns," in Proc. Int. Conf. Multimedia and Expo (ICME), 2007, pp.1091-1094.
  35. Gritti T., Shan C., Jeanne V., and Braspenning R., "Local features based facial expression recognition with face registration errors," in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition (FG), 2008.
  36. Hastie T., and Tibshirani R., and Friedman J., "The Elements of Statistical Learning; Data Mining, Inference, and Prediction", Springer, 2nd edition, 2009.

Cited by

  1. Facial Expression Recognition and Analysis: A Comparison Study of Feature Descriptors vol.7, pp.0, 2015, https://doi.org/10.2197/ipsjtcva.7.104
  2. Improvements in adhesion force and smart embedded programming of wall inspection robot vol.72, pp.7, 2016, https://doi.org/10.1007/s11227-015-1549-y
  3. A framework for unified digital evidence management in security convergence vol.13, pp.3, 2013, https://doi.org/10.1007/s10660-013-9119-y
  4. Mouse operation on monitor by interactive analysis of intuitive hand motions vol.75, pp.23, 2016, https://doi.org/10.1007/s11042-014-2357-8
  5. Event-Related Potentials Related to Anxiety in Emotion-Attention Interaction vol.2014, 2014, https://doi.org/10.1155/2014/408252
  6. Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform vol.75, pp.23, 2016, https://doi.org/10.1007/s11042-015-2455-2
  7. Design and implementation of the SARIMA–SVM time series analysis algorithm for the improvement of atmospheric environment forecast accuracy vol.22, pp.13, 2018, https://doi.org/10.1007/s00500-017-2825-y