KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Person-Independent Facial Expression Recognition with Histograms of Prominent Edge Directions

  • Makhmudkhujaev, Farkhod (Department of Computer Science and Engineering, Kyung Hee University) ;
  • Iqbal, Md Tauhid Bin (Department of Computer Science and Engineering, Kyung Hee University) ;
  • Arefin, Md Rifat (Department of Computer Science and Engineering, Kyung Hee University) ;
  • Ryu, Byungyong (Department of Computer Science and Engineering, Kyung Hee University) ;
  • Chae, Oksam (Department of Computer Science and Engineering, Kyung Hee University)
  • Received : 2018.04.24
  • Accepted : 2018.07.20
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a new descriptor, named Histograms of Prominent Edge Directions (HPED), for the recognition of facial expressions in a person-independent environment. In this paper, we raise the issue of sampling error in generating the code-histogram from spatial regions of the face image, as observed in the existing descriptors. HPED describes facial appearance changes based on the statistical distribution of the top two prominent edge directions (i.e., primary and secondary direction) captured over small spatial regions of the face. Compared to existing descriptors, HPED uses a smaller number of code-bins to describe the spatial regions, which helps avoid sampling error despite having fewer samples while preserving the valuable spatial information. In contrast to the existing Histogram of Oriented Gradients (HOG) that uses the histogram of the primary edge direction (i.e., gradient orientation) only, we additionally consider the histogram of the secondary edge direction, which provides more meaningful shape information related to the local texture. Experiments on popular facial expression datasets demonstrate the superior performance of the proposed HPED against existing descriptors in a person-independent environment.

Keywords

References

  1. C. Shan, S. Gong, and P. W. McOwan, "Facial expression recognition based on Local Binary Patterns: A comprehensive study," Image and Vision Computing, vol. 27, no. 6, pp. 803-816, May 2009. https://doi.org/10.1016/j.imavis.2008.08.005
  2. H. Hong, H. Neven, C. Von der Malsburg, "Online facial expression recognition based on personalized galleries," in Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 354-359, April 14-16, 1998.
  3. I. Kotsia and I. Pitas, "Facial expression recognition in image sequences using geometric deformation features and support vector machines," IEEE transactions on image processing, vol.16, no. 1, pp. 172-87, January 2007. https://doi.org/10.1109/TIP.2006.884954
  4. A. R. Rivera, J. A. R. Castillo, and O. Chae, "Local directional number pattern for face analysis: Face and expression recognition," IEEE transactions on image processing, vol. 22, no.5, pp. 1740-1752, May 2013. https://doi.org/10.1109/TIP.2012.2235848
  5. A. M. Martinez, A. C. Kak, "Pca versus lda," IEEE transactions on pattern analysis and machine intelligence, vol. 23, no. 2, pp. 228-33, February 2001. https://doi.org/10.1109/34.908974
  6. K. Etemad, R. Chellappa, "Discriminant analysis for recognition of human face images", JOSA A, vol. 14, no. 8, pp. 1724-33, August 1997. https://doi.org/10.1364/JOSAA.14.001724
  7. T. Jabid, Md. H. Kabir, and O. Chae, "Robust facial expression recognition based on local directional pattern," ETRI J., vol. 32, no. 5, pp. 784-794, October 2010. https://doi.org/10.4218/etrij.10.1510.0132
  8. A. R. Rivera, J. A. R. Castillo, and O. Chae, "Recognition of face expressions using local principal texture pattern," in Proc. of IEEE International Conference on Image Processing (ICIP), pp. 2613-2616, September 2012.
  9. M. T. B. Iqbal, B. Ryu, G. Song, and O. Chae, "Positional Ternary Pattern (PTP): An edge based image descriptor for human age recognition" in Proc. of IEEE International Conference on Consumer Electronics (ICCE), pp. 289-292, January 7-11, 2016.
  10. R. A. Kirsch, "Computer determination of the constituent structure of biological images," Computers and biomedical research, vol. 4, no. 3, pp. 315-328, 1971. https://doi.org/10.1016/0010-4809(71)90034-6
  11. M. T. B. Iqbal, M. Shoyaib, B. Ryu, M. Abdullah-Al-Wadud, O. Chae, "Directional Age-Primitive Pattern (DAPP) for Human Age Group Recognition and Age Estimation," IEEE Transactions on Information Forensics and Security, vol. 12, no. 11, pp. 2505-2517, November 2017. https://doi.org/10.1109/TIFS.2017.2695456
  12. P. Carcagnì, M. Coco, M. Leo, C. Distante, "Facial expression recognition and histograms of oriented gradients: a comprehensive study," SpringerPlus, vol. 4, no. 1, p. 645, December 2015. https://doi.org/10.1186/s40064-015-1427-3
  13. B. Ryu, A. R. Rivera, J. Kim, O. Chae, "Local directional ternary pattern for facial expression recognition," IEEE Transactions on Image Processing, vol. 26, no. 12, pp. 6006-6018, December 2017. https://doi.org/10.1109/TIP.2017.2726010
  14. G. S. Robinson, "Edge detection by compass gradient masks," Computer graphics and image processing, vol. 6, no. 5, pp. 492-501, October 1977. https://doi.org/10.1016/S0146-664X(77)80024-5
  15. P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews, "The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 94-101, June 13-18, 2010.
  16. N. C. Ebner, M. Riediger, and U. Lindenberger, "FACES-A database of facial expressions in young, middle-aged, and older women and men: Development and validation," Behavior Research Methods, vol. 42, no. 1, pp. 351-362, February 2010. https://doi.org/10.3758/BRM.42.1.351
  17. L. Yin, X. Wei, Y. Sun, J. Wang, M. J. Rosato, "A 3D facial expression database for facial behavior research," in Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 211-216, April 2, 2006.
  18. O. Langner, R. Dotsch, G. Bijlstra, D. H. Wigboldus, S. T. Hawk, A. D. Van Knippenberg, "Presentation and validation of the Radboud Faces Database," Cognition and Emotion, vol. 24, no. 8, pp. 1377-1388, December 2010. https://doi.org/10.1080/02699930903485076
  19. C. Cortes, V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-97, September 1995. https://doi.org/10.1007/BF00994018
  20. R. Ptucha, G. Tsagkatakis, A. Savakis, "Manifold based sparse representation for robust expression recognition without neutral subtraction," in Proc. of IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2136-2143, November 6, 2011.
  21. S. H. Lee, W. J. Baddar, Y. M. Ro, "Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos," Pattern Recognition, vol. 54, pp. 52-67, 2016. https://doi.org/10.1016/j.patcog.2015.12.016
  22. S. H. Lee, K. N. Plataniotis, Y. M. Ro, "Intra-class variation reduction using training expression images for sparse representation based facial expression recognition," IEEE Transactions on Affective Computing, vol. 5, pp. 340-51, 2014. https://doi.org/10.1109/TAFFC.2014.2346515
  23. B. F. Wu and C. H. Lin, "Adaptive Feature Mapping for Customizing Deep Learning Based Facial Expression Recognition Model," IEEE Access, vol.6, pp. 12451-12461, 2018. https://doi.org/10.1109/ACCESS.2018.2805861
  24. L. A. Jeni, D. Takacs and A. Lorincz, "High quality facial expression recognition in video streams using shape related information only," in Proc. of IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2168-2174, November 6, 2011.
  25. S. Yang, B. Bhanu, "Understanding discrete facial expressions in video using an emotion avatar image," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 4, pp. 980-92, August 2012. https://doi.org/10.1109/TSMCB.2012.2192269
  26. Z. Wang, S. Wang, Q. Ji, "Capturing complex spatio-temporal relations among facial muscles for facial expression recognition," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3422-3429, June 23, 2013.
  27. H. Jung, S. Lee, J. Yim, S. Park, J. Kim, "Joint fine-tuning in deep neural networks for facial expression recognition," in Proc. of IEEE International Conference on Computer Vision (ICCV), pp. 2983-2991, December 7, 2015.
  28. A. Caroppo, A. Leone, P. Siciliano, "Facial Expression Recognition in Older Adults using Deep Machine Learning," in Proc. of Third Italian Workshop on Artificial Intelligence for Ambient Assisted Living 2017, pp. 30-43, November 16-17, 2017.
  29. Y. Hu, Z. Zeng, L. Yin, X. Wei, J. Tu, T.S. Huang, "A study of non-frontal-view facial expressions recognition," in Proc. of International Conference on Pattern Recognition, pp. 1-4, December 8, 2008.
  30. W. Zheng, H. Tang, Z. Lin, T.S. Huang, "Emotion recognition from arbitrary view facial images," in Proc. of European Conference on Computer Vision, pp. 490-503, September 5, 2010.
  31. U. Tariq, T. S. Huang, "Features and fusion for expression recognition - A comparative analysis," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 146-152, June 16, 2012.
  32. S. Moore, R. Bowden, "Local binary patterns for multi-view facial expression recognition," Computer Vision and Image Understanding, vol. 115, no. 4, pp. 541-58, April, 2011. https://doi.org/10.1016/j.cviu.2010.12.001
  33. Z. Zhang, G. Lu, J. Yan, H. Li, N. Sun, X. Li, P. R. Zhenjiang, "Compact local Gabor directional number pattern for facial expression recognition," Turkish Journal of Electrical Engineering & Computer Sciences, vol. 26, no. 3, pp. 1236-1248, May 2018.
  34. M. T. Iqbal, M. Shoyaib, B. Ryu, M. Abdullah-Al-Wadud, O. Chae, "Directional age-primitive pattern (DAPP) for human age group recognition and age estimation," IEEE Transactions on Information Forensics and Security, vol. 12, no. 11, pp. 2505-2517, November 2017. https://doi.org/10.1109/TIFS.2017.2695456