KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multiscale Adaptive Local Directional Texture Pattern for Facial Expression Recognition

  • Zhang, Zhengyan (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Yan, Jingjie (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Lu, Guanming (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Li, Haibo (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Sun, Ning (Engineering Research Center of Wideband Wireless Communication Technology, Ministry of Education, Nanjing University of Posts and Telecommunications) ;
  • Ge, Qi (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
  • Received : 2016.12.07
  • Accepted : 2017.05.28
  • Published : 2017.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

This work presents a novel facial descriptor, which is named as multiscale adaptive local directional texture pattern (MALDTP) and employed for expression recognition. We apply an adaptive threshold value to encode facial image in different scales, and concatenate a series of histograms based on the MALDTP to generate facial descriptor in term of Gabor filters. In addition, some dedicated experiments were conducted to evaluate the performance of the MALDTP method in a person-independent way. The experimental results demonstrate that our proposed method achieves higher recognition rate than local directional texture pattern (LDTP). Moreover, the MALDTP method has lower computational complexity, fewer storage space and higher classification accuracy than local Gabor binary pattern histogram sequence (LGBPHS) method. In a nutshell, the proposed MALDTP method can not only avoid choosing the threshold by experience but also contain much more structural and contrast information of facial image than LDTP.

Keywords

References

  1. Mehrabian, A. and J.A. Russell, "An approach to environmental psychology," MIT Press, Cambridge, MA, USA, 1974.
  2. Khatri, N.N., Z.H. Shah, and S.A. Patel, "Facial expression recognition: A Survey," International Journal of Computer Science & Information Technologies, vol. 5, no. 1, pp. 149-152, Jan., 2014.
  3. Sumathi, C.P., T. Santhanam, and M. Mahadevi, "Automatic facial expression analysis a survey," International Journal of Computer Science & Engineering Survey, vol. 3, no. 6, pp. 47-59, Dec., 2012. https://doi.org/10.5121/ijcses.2012.3604
  4. Suthar, J. and N. Limbad, "A literature survey on facial expression recognition techniques using appearance based features," International Journal of Computer Trends & Technology, vol. 17, no. 4, pp. 161-165, Nov., 2014. https://doi.org/10.14445/22312803/IJCTT-V17P131
  5. Zavaschi, T.H.H., et al., "Fusion of feature sets and classifiers for facial expression recognition," Expert Systems with Applications, vol. 40, no. 2, pp. 646-655, Feb., 2013. https://doi.org/10.1016/j.eswa.2012.07.074
  6. Tian, Y., T. Kanade, and J.F. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Transactions on Pattern Analysis & Machine Intelligence. vol. 23, no. 2, pp. 97-115, Feb., 2001. https://doi.org/10.1109/34.908962
  7. Tian, Y., T. Kanade, and J.F. Cohn, "Facial expression recognition," in Proc. of Handbook of face recognition (Second Edition), Springer London, London, UK, pp. 487-519, 2011.
  8. Sung, E. and R.E. Mayer, "Five facets of social presence in online distance education," Computers in Human Behavior, vol. 28, no. 5, pp. 1738-1747, Sept., 2012. https://doi.org/10.1016/j.chb.2012.04.014
  9. Fasel, B. and J. Luettin, "Automatic facial expression analysis: a survey," Pattern Recognition, vol. 36, no. 1, pp. 259-275, Jan., 2003. https://doi.org/10.1016/S0031-3203(02)00052-3
  10. Samal, A. and P.A. Iyengar, "Automatic recognition and analysis of human faces and facial expressions: a survey," Pattern Recognition, vol. 25, no. 1, pp. 65-77, Jan., 1992. https://doi.org/10.1016/0031-3203(92)90007-6
  11. Li, Z., et al., "Robust Structured Subspace Learning for Data Representation," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 37, no.10, pp. 2085-2098, Oct., 2015. https://doi.org/10.1109/TPAMI.2015.2400461
  12. Z, L. and T. J, "Unsupervised Feature Selection Via Nonnegative Spectral Analysis and Redundancy Control," IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, vol. 24, no. 12, pp. 5343-5355, Dec., 2015. https://doi.org/10.1109/TIP.2015.2479560
  13. Youssif, A.A.A. and W.A.A. Asker, "Automatic facial expression recognition system based on geometric and appearance features," Computer & Information Science, vol. 4, no. 2, pp. 115-124, Mar., 2011.
  14. Kan, M., et al., "Multi-view discriminant analysis," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 38, no. 1, pp. 188-194, Jan., 2016. https://doi.org/10.1109/TPAMI.2015.2435740
  15. Peltonen, J. and S. Kaski, "Discriminative components of data," IEEE Transactions on Neural Networks, vol. 16, no. 1, pp. 68-83, Jan., 2005. https://doi.org/10.1109/TNN.2004.836194
  16. Lee, S.H., K.N. Kostas Plataniotis, and M.R. Yong, "Intra-class variation reduction using training expression images for sparse representation based facial expression recognition," IEEE Transactions on Affective Computing, vol. 5, no. 3, pp. 340-351, July-Sept., 2014. https://doi.org/10.1109/TAFFC.2014.2346515
  17. LeCun, Y., Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May, 2015. https://doi.org/10.1038/nature14539
  18. Schmidhuber, J., "Deep learning in neural networks: An overview," Neural Networks, vol. 61, pp. 85-117, Jan., 2014.
  19. Tang, Y., "Deep learning using linear support vector machines," in Proc. of International Conference on Machine Learning, Atlanta, USA, June 16-21, 2013.
  20. Bengio, Y., "Learning deep architectures for AI," Foundations and $trends^{(R)}$ in Machine Learning, vol. 2, no. 1, pp. 1-127, Jan., 2009. https://doi.org/10.1561/2200000006
  21. Cheng, Y., B. Jiang, and K. Jia, "A Deep Structure for Facial Expression Recognition under Partial Occlusion," in Proc. of Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kitakyushu, Japan, Aug. 27-29, 2014.
  22. Aroussi, Mohamed El, et al., "Local appearance based face recognition method using block based steerable pyramid transform," Signal Processing, vol. 91, no. 1, pp. 38-50, Jan., 2011. https://doi.org/10.1016/j.sigpro.2010.06.005
  23. Zhang, S., X. Zhao, and B. Lei, "Facial expression recognition based on local binary patterns and local fisher discriminant analysis," Wseas Transactions on Signal Processing, vol. 8, no. 1, pp. 21-31, Jan., 2012.
  24. Shan, C. and T. Gritti. "Learning Discriminative LBP-Histogram Bins for Facial Expression Recognition," in Proc. of British Machine Vision Conference, Leeds, UK, Sept. 1-4, 2008.
  25. Zhang, Z., "Feature-based facial expression recognition: sensitivity analysis and experiments with a multi-layer perceptron," International Journal of Pattern Recognition & Artificial Intelligence, vol. 13, no. 6, pp. 893-911, Sept., 2011.
  26. Ojala, T., M. Pietikainen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern recognition, vol. 29, no. 1, pp. 51-59, Jan., 1996. https://doi.org/10.1016/0031-3203(95)00067-4
  27. Ojala, T., M. Pietikainen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," in Proc. of 12th International Conference on Pattern Recognition, vol. 1, pp. 582-585, Jerusalem, Israel, Oct. 9-13, 1994.
  28. Ojala, T., M. Pietikainen, and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on pattern analysis and machine intelligence. vol. 24, no. 7, pp. 971-987, July, 2002. https://doi.org/10.1109/TPAMI.2002.1017623
  29. Shan, C., S. Gong, and P.W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study," Image and Vision Computing, vol. 27, no. 6, pp. 803-816, May, 2009. https://doi.org/10.1016/j.imavis.2008.08.005
  30. Zhang, S., X. Zhao, and B. Lei, "Facial expression recognition based on local binary patterns and local fisher discriminant analysis," WSEAS Transactions on Signal Process, vol. 8, no. 1, pp. 21-31, Jan., 2012.
  31. Hablani, R., N. Chaudhari, and S. Tanwani, "Recognition of facial expressions using local binary patterns of important facial parts," International Journal of Image Processing (IJIP), vol. 7, no. 2, pp. 163-170, Apr., 2013.
  32. Shan, C., S. Gong, and P.W. McOwan, "Robust facial expression recognition using local binary patterns," in Proc. of IEEE International Conference on Image Processing, Genoa, Italy, Sept. 11-14, 2005.
  33. Zhou, H., R. Wang, and C. Wang, "A novel extended local-binary-pattern operator for texture analysis," Information Sciences, vol. 178, no. 22, pp. 4314-4325, Nov, 2008. https://doi.org/10.1016/j.ins.2008.07.015
  34. Hasanul Kabir, T.J. and O. Chae, "Local directional pattern variance (LDPv): A robust feature descriptor for facial expression recognition," International Arab Journal of Information Technology, vol. 9, no. 4, pp. 382-391, July, 2010.
  35. Jabid, T., M.H. Kabir, and O. Chae, "Local directional pattern (LDP) for face recognition," in Proc. of the IEEE International Conference on Consumer Electronics, pp. 329 - 330, Nha Trang, VIETANAM, Aug. 11-13, 2010.
  36. Jabid, T., M.H. Kabir, and O. Chae, "Facial expression recognition using local directional pattern (LDP)," in Proc. of IEEE International Conference on Image Processing, Hong Kong, China, Sept. 26-29, 2010.
  37. Lyons, M., et al., "Coding facial expressions with gabor wavelets," in Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200-205, Nara, Japan, Apr. 14-16, 1998.
  38. Deng, H.-B., et al., "A new facial expression recognition method based on local gabor filter bank and pca plus lda," International Journal of Information Technology, vol. 11, no. 11, pp. 86-96, Nov., 2005.
  39. Lyons, M.J., J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, Dec., 1999. https://doi.org/10.1109/34.817413
  40. Tai, S.L., "Image Representation Using 2D Gabor Wavelets," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 18, no. 10, pp. 959-971, Oct., 1996. https://doi.org/10.1109/34.541406
  41. Shen, L. and L. Bai, "MutualBoost learning for selecting Gabor features for face recognition," Pattern Recognition Letters, vol. 27, no. 15, pp. 1758-1767, Nov., 2006. https://doi.org/10.1016/j.patrec.2006.02.005
  42. Zhang, W., et al., "Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition," in Proc. of IEEE International Conference on Computer Vision, vol. 1, no. 786-791, Beijing, China, Oct. 17-20, 2005.
  43. Rivera, A.R., J.R. Castillo, and O. Chae, "Local directional texture pattern image descriptor," Pattern Recognition Letters. vol. 51, no. 1, pp. 94-100, Jan., 2015. https://doi.org/10.1016/j.patrec.2014.08.012
  44. Lucey, P., et al., "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 94 - 101, San Francisco, California, USA, June 13-18, 2010.
  45. Kamarainen, J., V. Kyrki, and H. Kalviainen, "Fundamental frequency Gabor filters for object recognition," in Proc. of the 16th International Conference on Pattern Recognition, vol.1, pp. 628-631, Quebec, Canada, Aug. 11-15, 2002.
  46. STruc, V., and Pave, Nikola, "Gabor-based kernel partial-least-squares discrimination features for face recognition," Informatica, vol. 20, no. 1, pp. 115-138, Jan., 2009.
  47. Haghighat, M., S. Zonouz, and M. Abdel-Mottaleb, "CloudID: Trustworthy cloud-based and cross-enterprise biometric identification," Expert Systems with Applications, vol. 42, no. 21, pp. 7905-7916, Nov., 2015. https://doi.org/10.1016/j.eswa.2015.06.025
  48. Shen, L. and L. Bai, "A review on Gabor wavelets for face recognition," Pattern Analysis and Applications, vol. 9, no. 2, pp. 273-292, Oct., 2006. https://doi.org/10.1007/s10044-006-0033-y
  49. Zeng, H., et al., "Compact Local Directional Texture Pattern for local image description," Advances in Multimedia, vol. 2015, no. 8, pp. 1-10, Setp., 2015.
  50. Viola, P. and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1-I, pp. I-511-518, Kauai, HI, USA, Dec. 8-14, 2001.
  51. Chang, C.-C. and C.-J. Lin, "LIBSVM: a library for support vector machines," ACM Transactions on Intelligent Systems and Technology (TIST), vol. 2, no. 3, pp. 389-396, Apr., 2011.
  52. Xue, M., W. Liu, and L. Li, "Person-independent facial expression recognition via hierarchical classification," in Proc. of IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 449 - 454, Melbourne, Australia, Apr. 2-5, 2013.
  53. Du, Y. and X. Lin, "Mapping Emotional Status to Facial Expressions," in Proc. of 16th International Conference on Pattern Recognition, vol. 2, pp. 524- 527, Quebec, Canada, Aug. 11-15, 2002.
  54. Du, Y. and X. Lin, "Emotional facial expression model building," Pattern Recognition Letters. vol. 24, no. 16, pp. 2923-2934, Dec., 2003. https://doi.org/10.1016/S0167-8655(03)00153-3
  55. Abdi, H. and L.J. Williams, "Principal component analysis," Wiley Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, pp. 433-459, July, 2010. https://doi.org/10.1002/wics.101