Browse > Article

Study of Facial Expression Recognition using Variable-sized Block  

Cho, Youngtak (경희대학교 컴퓨터공학과)
Ryu, Byungyong (포스코ICT)
Chae, Oksam (경희대학교 컴퓨터공학과)
Publication Information
Convergence Security Journal / v.19, no.1, 2019 , pp. 67-78 More about this Journal
Abstract
Most existing facial expression recognition methods use a uniform grid method that divides the entire facial image into uniform blocks when describing facial features. The problem of this method may include non-face backgrounds, which interferes with discrimination of facial expressions, and the feature of a face included in each block may vary depending on the position, size, and orientation of the face in the input image. In this paper, we propose a variable-size block method which determines the size and position of a block that best represents meaningful facial expression change. As a part of the effort, we propose the way to determine the optimal number, position and size of each block based on the facial feature points. For the evaluation of the proposed method, we generate the facial feature vectors using LDTP and construct a facial expression recognition system based on SVM. Experimental results show that the proposed method is superior to conventional uniform grid based method. Especially, it shows that the proposed method can adapt to the change of the input environment more effectively by showing relatively better performance than exiting methods in the images with large shape and orientation changes.
Keywords
Facial Expression Recognition; LBP; LDTP; SVM; Variable-sized block;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 P. Weinberg, and W. Gottwald, "Impulsive consumer buying as a result of emotions", Journal of Business research, Vol. 10, No. 1, pp. 43-57, 1982.   DOI
2 D. Consoli, "Emotions that influence purchase decisions and their electronic processing", Annales Universitatis Apulensis: Series Oeconomica, Vol. 11, No. 2, pp. 996, 2009.
3 M. Kang, "A Study on Efficient Facial Expression Recognition System for Customer Satisfaction Feedback", Journal of Information and Security, Vol. 12, No. 4, pp. 75-84, Sept. 2012.
4 S. Huh, D. Lee, K. Kim, "A Study on The Improvement of User Authentication using the Facial Recognition and OTP Technique in the Mobile Environment", Journal of Information and Security, Vol. 11, No. 3, pp. 75-84, 2011.
5 Jai-Suk Lee, et al., "Research of video based Vibraimage technology stimulation examination KOCOSA", Journal of Information and Security, Vol. 15, Vo. 3, pp. 41-51, 2015.
6 Tian, Ying-Li, Takeo Kanade, and Jeffrey F. Cohn, "Facial expression analysis", Handbook of face recognition. Springer, New York, NY, pp. 247-275, 2005.
7 P. Belhumeur and J. P. Hespanha and D. J. Kriegman, "Eigenfaces vs. sherfaces: Recognition using class specific linear projection", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 7, pp. 711-720, 1997.
8 I. Kotsia, and I. Pitas, "Facial expression recognition in image sequences using geometric deformation features and support vector machines", IEEE trans. on image processing, Vol. 16, No. 1, pp. 172-187, 2007.   DOI
9 Pantic, Maja, and Marian Stewart Bartlett, "Machine analysis of facial expressions", Face recognition. InTech, 2007.
10 M. A. Turk and A. P. Pentland, "Face recognition using eigenfaces", Computer Vision and Pattern Recognition, Proceedings CVPR'91., IEEE Computer Society Conference on. IEEE, pp. 586-591, 1991.
11 M. Lyons, et al., "Coding facial expressions with gabor wavelets", Automatic Face and Gesture Recognition, 1998. Proceedings. 3rd IEEE Int'l Conference on. 1998.
12 Z. Nenadic, "Information discriminant analysis: Feature extraction with an information-theoretic objective", IEEE Trans on Pattern Analysis and Machine Intelligence, Vol. 29, No. 8, pp. 1394-1407, 2007.   DOI
13 J. Yang, et al., "Two-dimensional PCA: a new approach to appearance-based face representation and recognition", IEEE trans. on pattern analysis and machine intelligence, Vol. 26, No. 1, pp. 131-137, 2004.   DOI
14 T. Ahonen, A. Hadid, and M. Pietikainen, "Face description with local binary patterns: Application to face recognition", IEEE Trans. on Pattern Analysis & Machine Intelligence, Vol. 12, pp. 2037-2041, 2006.
15 Jabid, Taskeed, Md Hasanul Kabir, and Oksam Chae. "Robust facial expression recognition based on local directional pattern." ETRI journal, Vol. 32, No. 5 pp. 784-794, 2010.   DOI
16 Paul, Ekman, Wallace V. Friesen, and Joseph C. Hager, "Facial Action Coding System: the manual on CD ROM", Salt Lake City, A Human Face, 2002.
17 C. Shan, S. Gong, and P. W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study", Image and vision Computing, Vol. 27, No. 6, pp. 803-816, 2009.   DOI
18 Z. Niu, et al., "2d cascaded adaboost for eye localization", Pattern Recognition, 2006. ICPR 2006. 18th International Conference on. Vol. 2. IEEE, pp. 1216-1219, 2006.
19 Tzimiropoulos, Georgios, and Maja Pantic, "Fast algorithms for fitting active appearance models to unconstrained images", International journal of computer vision, Vol 122, No. 1, pp. 17-33, 2017.   DOI
20 A. Asthana, et al., "Incremental face alignment in the wild", Proc. of the IEEE conf. on CVPR, pp. 1859-1866, 2014.
21 P. Ekman, E. Rosenberg, and J. Hager, Facial action coding system affect interpretation dictionary (facsaid), 1998 (cit. on pp. 6, 29, 38, 55).
22 B. Ryu, et al., "Local directional ternary pattern for facial expression recognition", IEEE Trans. on Image Processing, Vol. 26, No. 12, pp. 6006-6018, 2017.   DOI
23 C. Cortes, and V. Vapnik, "Support-vector net works", Machine learning Vol. 20, No. 3, pp. 273-297, 1995.   DOI
24 Lucey, Patrick, et al., "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression", Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, pp. 94-101, 2010.
25 M. Valstar, and M. Pantic, "Induced disgust, happiness and surprise: an addition to the mmi facial expression database", Proc. 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect. pp. 65, 2010.
26 A. Mollahosseini, D. Chan and M. H. Mahoor, "Going deeper in facial expression recognition using deep neural networks", 2016 IEEE winter conference on applications of computer vision (WACV), IEEE, pp. 1-10, 2016.
27 S. W. Chew, et al., "Person-independent facial expression detection using constrained local models", Face and Gesture 2011. IEEE, pp. 915-920, 2011.
28 L. A. Jeni, D. Takacs and A. Lorincz, "High quality facial expression recognition in video streams using shape related information only", 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), IEEE, pp. 2168-2174, 2011.
29 S. Yang and B. Bhanu, "Understanding discrete facial expressions in video using an emotion avatar image", IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 42, No. 4, pp. 980-992, 2012.   DOI
30 S. Lee, K. N. K. Plataniotis and Y. Ro, "Intra-class variation reduction using training expression images for sparse representation based facial expression recognition", IEEE Transactions on Affective Computing, Vol. 5, No. 3, pp. 340-351, 2014.   DOI
31 M. S. Bartlett, et al., "Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction", 2003 Computer Vision and Pattern Recognition Workshop, Conference on, IEEE, Vol. 5, pp. 53-53, 2003.
32 L. Zhong, et al., "Learning active facial patches for expression analysis", 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 2562-2569, 2012.
33 J. Wang and L. Yin, "Static topographic modeling for facial expression recognition and analysis", Computer Vision and Image Understanding, Vol. 108, No.1-2, pp. 19-34, 2007.   DOI
34 P. Burkert, et al., "Dexpression: Deep convolutional neural network for expression recognition", arXiv preprint arXiv:1509.05371, 2015.