KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-classifier Fusion Based Facial Expression Recognition Approach

  • Jia, Xibin (Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology) ;
  • Zhang, Yanhua (Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology) ;
  • Powers, David (Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology) ;
  • Ali, Humayra Binte (School of Computer Science, Engineering and Mathematics, Flinders University of South Australia)
  • Received : 2013.10.30
  • Accepted : 2014.12.23
  • Published : 2014.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

Facial expression recognition is an important part in emotional interaction between human and machine. This paper proposes a facial expression recognition approach based on multi-classifier fusion with stacking algorithm. The kappa-error diagram is employed in base-level classifiers selection, which gains insights about which individual classifier has the better recognition performance and how diverse among them to help improve the recognition accuracy rate by fusing the complementary functions. In order to avoid the influence of the chance factor caused by guessing in algorithm evaluation and get more reliable awareness of algorithm performance, kappa and informedness besides accuracy are utilized as measure criteria in the comparison experiments. To verify the effectiveness of our approach, two public databases are used in the experiments. The experiment results show that compared with individual classifier and two other typical ensemble methods, our proposed stacked ensemble system does recognize facial expression more accurately with less standard deviation. It overcomes the individual classifier's bias and achieves more reliable recognition results.

Keywords

References

  1. Tariq, Usman, and Thomas S. Huang, "Features and fusion for expression recognition-A comparative analysis," in Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, pp.146-152, June 16-21, 2012.
  2. Ekman, Paul, and Wallace V. Friesen, "Constants across cultures in the face and emotion," Journal of personality and social psychology, vol. 17, no.2, pp.124-129, February, 1971. https://doi.org/10.1037/h0030377
  3. P. Ekman, W. Friesen, and J. Hager, "The Facial Action Coding System: The Manual on CD ROM," A Human Face, Salt Lake City, 2002.
  4. Lanitis, Andreas, Christopher J, Taylor, and Timothy F. Cootes. "Automatic interpretation and coding of face images using flexible models," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 19, no.7, pp. 743-756, July, 1997. https://doi.org/10.1109/34.598231
  5. Zhang, Yongmian, and Qiang Ji, "Active and dynamic information fusion for facial expression understanding from image sequences," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.27, no.5, pp.699-714, May, 2005. https://doi.org/10.1109/TPAMI.2005.93
  6. Shan, Caifeng, Shaogang Gong, and Peter W. McOwan. "Facial expression recognition based on Local Binary Patterns: A comprehensive study," Image and Vision Computing, vol.27, no.6, pp. 803-816, May, 2009. https://doi.org/10.1016/j.imavis.2008.08.005
  7. Yang P, Liu Q, Metaxas D N, "Exploring facial expressions with compositional features," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.2638-2644, June 13-18, 2010.
  8. Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. "Recognizing facial expression: Machine learning and application to spontaneous behavior," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.568-573, June 20-25, 2005.
  9. Koelstra Sander, Maja Pantic, and Ioannis Patras, "A dynamic texture-based approach to recognition of facial actions and their temporal models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.32, no.11, pp.1940-1954, November, 2010. https://doi.org/10.1109/TPAMI.2010.50
  10. Thiago H.H. Zavaschi, Alceu S. Britto Jr., Luiz E.S. Oliveira, Alessandro L. Koerich, "Fusion of feature sets and classifiers for facial expression recognition," Expert Systems with Applications, vol.40, no.2, pp.646-655, February, 2013. https://doi.org/10.1016/j.eswa.2012.07.074
  11. David H. Wolpert, "Stacked generalization," Neural Networks, vol.5, no.2, pp.241-259, March, 1992. https://doi.org/10.1016/S0893-6080(05)80023-1
  12. Sulzmann J N, Furnkranz J, "Rule Stacking: An approach for compressing an ensemble of rule sets into a single classifier," Discovery Science, Springer Berlin Heidelberg, pp.323-334, 2011.
  13. D.D. Margineantu and T.G. Dietterich, "Pruning Adaptive Boosting," in Proc. of 14th Int'l Conf. Machine Learning, vol.97, pp.211-218, July, 1997.
  14. J. Kittler, M. Hatef, R.P.W. Duin, J. Matas, "On combining classifiers," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no.3, pp.226-239, March, 1998. https://doi.org/10.1109/34.667881
  15. Li Shoushan, Huang ChuRan, "Chinese Sentiment Classifi-cation Based on Stacking Combination Method," Journal of Chinese Information Processing, vol.24, no.5, pp.56-61, November, 2010.
  16. Koutlas, A., & Fotiadis, D. I., "An automatic region based methodology for facial expression recognition," in Proc. of IEEE Conf. on Systems, Man and Cybernetics, pp. 662-666, October 12-15, 2008.
  17. Yong Xu, Qi Zhu, Zizhu Fan, Minna Qiu, Yan Chen, Hong Liu, "Coarse to fine K nearest neighbor classifier, " Pattern Recognition Letters, vol.34, no.9, pp.980-986, July, 2013. https://doi.org/10.1016/j.patrec.2013.01.028
  18. Ludmila I. Kuncheva, "A Bound on Kappa-Error Diagrams for Analysis of Classifier Ensembles," IEEE Transaction on Knowledge and Data Engineering, vol.25, no.3, pp.494-501, March, 2013. https://doi.org/10.1109/TKDE.2011.234
  19. Platt, John, "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines", April, 1998.
  20. Kuncheva L I, "Combining Pattern Classifiers: Methods and Algorithms (Kuncheva, LI; 2004)[book review]," Neural Networks, IEEE Transactions on, vol.18, no.3, pp.964-964, May, 2007.
  21. Ting, K. M., Witten, I. H., "Stacking Bagged and Dagged Models," in Proc. of 14th Conf. on Machine Learning, pp.367-375, March, 1997.
  22. Zhong L, Liu Q, Yang P, et al, "Learning active facial patches for expression analysis", in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.2562-2569, June 16-21, 2012.
  23. Cohen, "A coefficient of agreement for nominal scales," Educational and Psychological Measurement, vol.20, no.1, pp.37- 46, April, 1960. https://doi.org/10.1177/001316446002000104
  24. Powers, D. M. W. "Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation," Journal of Machine Learning Technologies, vol.2, no.1, pp.37-63, February, 2011.
  25. Breiman, L., "Bagging Predictors," Machine Learning, vol.24, no.2, pp.123-140, August, 1996.
  26. Cohen, I., Sebe, N., Garg, A., Chen, L., & Huang, T. S. "Facial expression recognition from video sequences: Temporal and static modeling," Computer Vision and Image Understanding, vol.91, no.1, pp.160-187, July, 2003. https://doi.org/10.1016/S1077-3142(03)00081-X
  27. Shan, C., Gong, S., McOwan, P.W., "Robust facial expression recognition using local binary patterns," in Proc. of IEEE Conf. on Image Processing, pp.370-373, September 11-14, 2005.
  28. Bashyal, S., & Venayagamoorthy, G. K., "Recognition of facial expressions using gabor wavelets and learning vector quantization," Engineering Applications of Artificial Intelligence, vol.21, no.7, pp.1056-1064, October, 2008. https://doi.org/10.1016/j.engappai.2007.11.010
  29. Kaimin Yu, Zhiyong Wang, Li Zhuo, Jiajun Wang, Zheru Chi, Dagan Feng, "Learning realistic facial expressions from web images," Pattern Recognition, vol.46, no.8, pp.2144-2155, August, 2013. https://doi.org/10.1016/j.patcog.2013.01.032

Cited by

  1. A Multi-Layer Fusion-Based Facial Expression Recognition Approach with Optimal Weighted AUs vol.7, pp.2, 2014, https://doi.org/10.3390/app7020112
  2. MRMR-based ensemble pruning for facial expression recognition vol.77, pp.12, 2014, https://doi.org/10.1007/s11042-017-5105-z
  3. RTCRelief-F: an effective clustering and ordering-based ensemble pruning algorithm for facial expression recognition vol.59, pp.1, 2014, https://doi.org/10.1007/s10115-018-1176-z
  4. Graph-based dynamic ensemble pruning for facial expression recognition vol.49, pp.9, 2014, https://doi.org/10.1007/s10489-019-01435-2