Browse > Article
http://dx.doi.org/10.3837/tiis.2017.09.020

Multiscale Adaptive Local Directional Texture Pattern for Facial Expression Recognition  

Zhang, Zhengyan (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
Yan, Jingjie (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
Lu, Guanming (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
Li, Haibo (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
Sun, Ning (Engineering Research Center of Wideband Wireless Communication Technology, Ministry of Education, Nanjing University of Posts and Telecommunications)
Ge, Qi (College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.11, no.9, 2017 , pp. 4549-4566 More about this Journal
Abstract
This work presents a novel facial descriptor, which is named as multiscale adaptive local directional texture pattern (MALDTP) and employed for expression recognition. We apply an adaptive threshold value to encode facial image in different scales, and concatenate a series of histograms based on the MALDTP to generate facial descriptor in term of Gabor filters. In addition, some dedicated experiments were conducted to evaluate the performance of the MALDTP method in a person-independent way. The experimental results demonstrate that our proposed method achieves higher recognition rate than local directional texture pattern (LDTP). Moreover, the MALDTP method has lower computational complexity, fewer storage space and higher classification accuracy than local Gabor binary pattern histogram sequence (LGBPHS) method. In a nutshell, the proposed MALDTP method can not only avoid choosing the threshold by experience but also contain much more structural and contrast information of facial image than LDTP.
Keywords
Facial expression recognition; multiscale adaptive local directional texture pattern; facial descriptor; Gabor filters;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Shen, L. and L. Bai, "MutualBoost learning for selecting Gabor features for face recognition," Pattern Recognition Letters, vol. 27, no. 15, pp. 1758-1767, Nov., 2006.   DOI
2 Zhang, W., et al., "Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition," in Proc. of IEEE International Conference on Computer Vision, vol. 1, no. 786-791, Beijing, China, Oct. 17-20, 2005.
3 Rivera, A.R., J.R. Castillo, and O. Chae, "Local directional texture pattern image descriptor," Pattern Recognition Letters. vol. 51, no. 1, pp. 94-100, Jan., 2015.   DOI
4 Lucey, P., et al., "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 94 - 101, San Francisco, California, USA, June 13-18, 2010.
5 Kamarainen, J., V. Kyrki, and H. Kalviainen, "Fundamental frequency Gabor filters for object recognition," in Proc. of the 16th International Conference on Pattern Recognition, vol.1, pp. 628-631, Quebec, Canada, Aug. 11-15, 2002.
6 STruc, V., and Pave, Nikola, "Gabor-based kernel partial-least-squares discrimination features for face recognition," Informatica, vol. 20, no. 1, pp. 115-138, Jan., 2009.
7 Haghighat, M., S. Zonouz, and M. Abdel-Mottaleb, "CloudID: Trustworthy cloud-based and cross-enterprise biometric identification," Expert Systems with Applications, vol. 42, no. 21, pp. 7905-7916, Nov., 2015.   DOI
8 Shen, L. and L. Bai, "A review on Gabor wavelets for face recognition," Pattern Analysis and Applications, vol. 9, no. 2, pp. 273-292, Oct., 2006.   DOI
9 Zeng, H., et al., "Compact Local Directional Texture Pattern for local image description," Advances in Multimedia, vol. 2015, no. 8, pp. 1-10, Setp., 2015.
10 Cheng, Y., B. Jiang, and K. Jia, "A Deep Structure for Facial Expression Recognition under Partial Occlusion," in Proc. of Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kitakyushu, Japan, Aug. 27-29, 2014.
11 Aroussi, Mohamed El, et al., "Local appearance based face recognition method using block based steerable pyramid transform," Signal Processing, vol. 91, no. 1, pp. 38-50, Jan., 2011.   DOI
12 Zhang, S., X. Zhao, and B. Lei, "Facial expression recognition based on local binary patterns and local fisher discriminant analysis," Wseas Transactions on Signal Processing, vol. 8, no. 1, pp. 21-31, Jan., 2012.
13 Shan, C. and T. Gritti. "Learning Discriminative LBP-Histogram Bins for Facial Expression Recognition," in Proc. of British Machine Vision Conference, Leeds, UK, Sept. 1-4, 2008.
14 Zhang, Z., "Feature-based facial expression recognition: sensitivity analysis and experiments with a multi-layer perceptron," International Journal of Pattern Recognition & Artificial Intelligence, vol. 13, no. 6, pp. 893-911, Sept., 2011.
15 Ojala, T., M. Pietikainen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern recognition, vol. 29, no. 1, pp. 51-59, Jan., 1996.   DOI
16 Xue, M., W. Liu, and L. Li, "Person-independent facial expression recognition via hierarchical classification," in Proc. of IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 449 - 454, Melbourne, Australia, Apr. 2-5, 2013.
17 Ojala, T., M. Pietikainen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," in Proc. of 12th International Conference on Pattern Recognition, vol. 1, pp. 582-585, Jerusalem, Israel, Oct. 9-13, 1994.
18 Ojala, T., M. Pietikainen, and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on pattern analysis and machine intelligence. vol. 24, no. 7, pp. 971-987, July, 2002.   DOI
19 Viola, P. and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1-I, pp. I-511-518, Kauai, HI, USA, Dec. 8-14, 2001.
20 Chang, C.-C. and C.-J. Lin, "LIBSVM: a library for support vector machines," ACM Transactions on Intelligent Systems and Technology (TIST), vol. 2, no. 3, pp. 389-396, Apr., 2011.
21 Du, Y. and X. Lin, "Mapping Emotional Status to Facial Expressions," in Proc. of 16th International Conference on Pattern Recognition, vol. 2, pp. 524- 527, Quebec, Canada, Aug. 11-15, 2002.
22 Du, Y. and X. Lin, "Emotional facial expression model building," Pattern Recognition Letters. vol. 24, no. 16, pp. 2923-2934, Dec., 2003.   DOI
23 Abdi, H. and L.J. Williams, "Principal component analysis," Wiley Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, pp. 433-459, July, 2010.   DOI
24 Khatri, N.N., Z.H. Shah, and S.A. Patel, "Facial expression recognition: A Survey," International Journal of Computer Science & Information Technologies, vol. 5, no. 1, pp. 149-152, Jan., 2014.
25 Shan, C., S. Gong, and P.W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study," Image and Vision Computing, vol. 27, no. 6, pp. 803-816, May, 2009.   DOI
26 Zhang, S., X. Zhao, and B. Lei, "Facial expression recognition based on local binary patterns and local fisher discriminant analysis," WSEAS Transactions on Signal Process, vol. 8, no. 1, pp. 21-31, Jan., 2012.
27 Mehrabian, A. and J.A. Russell, "An approach to environmental psychology," MIT Press, Cambridge, MA, USA, 1974.
28 Zavaschi, T.H.H., et al., "Fusion of feature sets and classifiers for facial expression recognition," Expert Systems with Applications, vol. 40, no. 2, pp. 646-655, Feb., 2013.   DOI
29 Sumathi, C.P., T. Santhanam, and M. Mahadevi, "Automatic facial expression analysis a survey," International Journal of Computer Science & Engineering Survey, vol. 3, no. 6, pp. 47-59, Dec., 2012.   DOI
30 Suthar, J. and N. Limbad, "A literature survey on facial expression recognition techniques using appearance based features," International Journal of Computer Trends & Technology, vol. 17, no. 4, pp. 161-165, Nov., 2014.   DOI
31 Tian, Y., T. Kanade, and J.F. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Transactions on Pattern Analysis & Machine Intelligence. vol. 23, no. 2, pp. 97-115, Feb., 2001.   DOI
32 Tian, Y., T. Kanade, and J.F. Cohn, "Facial expression recognition," in Proc. of Handbook of face recognition (Second Edition), Springer London, London, UK, pp. 487-519, 2011.
33 Sung, E. and R.E. Mayer, "Five facets of social presence in online distance education," Computers in Human Behavior, vol. 28, no. 5, pp. 1738-1747, Sept., 2012.   DOI
34 Z, L. and T. J, "Unsupervised Feature Selection Via Nonnegative Spectral Analysis and Redundancy Control," IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, vol. 24, no. 12, pp. 5343-5355, Dec., 2015.   DOI
35 Fasel, B. and J. Luettin, "Automatic facial expression analysis: a survey," Pattern Recognition, vol. 36, no. 1, pp. 259-275, Jan., 2003.   DOI
36 Samal, A. and P.A. Iyengar, "Automatic recognition and analysis of human faces and facial expressions: a survey," Pattern Recognition, vol. 25, no. 1, pp. 65-77, Jan., 1992.   DOI
37 Li, Z., et al., "Robust Structured Subspace Learning for Data Representation," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 37, no.10, pp. 2085-2098, Oct., 2015.   DOI
38 Youssif, A.A.A. and W.A.A. Asker, "Automatic facial expression recognition system based on geometric and appearance features," Computer & Information Science, vol. 4, no. 2, pp. 115-124, Mar., 2011.
39 Kan, M., et al., "Multi-view discriminant analysis," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 38, no. 1, pp. 188-194, Jan., 2016.   DOI
40 Peltonen, J. and S. Kaski, "Discriminative components of data," IEEE Transactions on Neural Networks, vol. 16, no. 1, pp. 68-83, Jan., 2005.   DOI
41 Lee, S.H., K.N. Kostas Plataniotis, and M.R. Yong, "Intra-class variation reduction using training expression images for sparse representation based facial expression recognition," IEEE Transactions on Affective Computing, vol. 5, no. 3, pp. 340-351, July-Sept., 2014.   DOI
42 Jabid, T., M.H. Kabir, and O. Chae, "Facial expression recognition using local directional pattern (LDP)," in Proc. of IEEE International Conference on Image Processing, Hong Kong, China, Sept. 26-29, 2010.
43 Hablani, R., N. Chaudhari, and S. Tanwani, "Recognition of facial expressions using local binary patterns of important facial parts," International Journal of Image Processing (IJIP), vol. 7, no. 2, pp. 163-170, Apr., 2013.
44 Shan, C., S. Gong, and P.W. McOwan, "Robust facial expression recognition using local binary patterns," in Proc. of IEEE International Conference on Image Processing, Genoa, Italy, Sept. 11-14, 2005.
45 Zhou, H., R. Wang, and C. Wang, "A novel extended local-binary-pattern operator for texture analysis," Information Sciences, vol. 178, no. 22, pp. 4314-4325, Nov, 2008.   DOI
46 Hasanul Kabir, T.J. and O. Chae, "Local directional pattern variance (LDPv): A robust feature descriptor for facial expression recognition," International Arab Journal of Information Technology, vol. 9, no. 4, pp. 382-391, July, 2010.
47 Jabid, T., M.H. Kabir, and O. Chae, "Local directional pattern (LDP) for face recognition," in Proc. of the IEEE International Conference on Consumer Electronics, pp. 329 - 330, Nha Trang, VIETANAM, Aug. 11-13, 2010.
48 Lyons, M., et al., "Coding facial expressions with gabor wavelets," in Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200-205, Nara, Japan, Apr. 14-16, 1998.
49 Lyons, M.J., J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, Dec., 1999.   DOI
50 Deng, H.-B., et al., "A new facial expression recognition method based on local gabor filter bank and pca plus lda," International Journal of Information Technology, vol. 11, no. 11, pp. 86-96, Nov., 2005.
51 Tai, S.L., "Image Representation Using 2D Gabor Wavelets," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 18, no. 10, pp. 959-971, Oct., 1996.   DOI
52 Bengio, Y., "Learning deep architectures for AI," Foundations and $trends^{(R)}$ in Machine Learning, vol. 2, no. 1, pp. 1-127, Jan., 2009.   DOI
53 LeCun, Y., Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May, 2015.   DOI
54 Schmidhuber, J., "Deep learning in neural networks: An overview," Neural Networks, vol. 61, pp. 85-117, Jan., 2014.
55 Tang, Y., "Deep learning using linear support vector machines," in Proc. of International Conference on Machine Learning, Atlanta, USA, June 16-21, 2013.