Browse > Article
http://dx.doi.org/10.5909/JBE.2017.22.2.162

Facial Expression Classification Using Deep Convolutional Neural Network  

Choi, In-kyu (Department of Electrical Engineering, KwangWoon University)
Song, Hyok (Department of Electronic Engineering)
Lee, Sangyong (Department of Electrical Engineering, KwangWoon University)
Yoo, Jisang (Department of Electrical Engineering, KwangWoon University)
Publication Information
Journal of Broadcast Engineering / v.22, no.2, 2017 , pp. 162-172 More about this Journal
Abstract
In this paper, we propose facial expression recognition using CNN (Convolutional Neural Network), one of the deep learning technologies. To overcome the disadvantages of existing facial expression databases, various databases are used. In the proposed technique, we construct six facial expression data sets such as 'expressionless', 'happiness', 'sadness', 'angry', 'surprise', and 'disgust'. Pre-processing and data augmentation techniques are also applied to improve efficient learning and classification performance. In the existing CNN structure, the optimal CNN structure that best expresses the features of six facial expressions is found by adjusting the number of feature maps of the convolutional layer and the number of fully-connected layer nodes. Experimental results show that the proposed scheme achieves the highest classification performance of 96.88% while it takes the least time to pass through the CNN structure compared to other models.
Keywords
Convolutional neural network; face expression; data augmentation; data-set;
Citations & Related Records
연도 인용수 순위
  • Reference
1 K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," arXiv preprint arXiv:1512.03385, 2015.
2 Mollahosseini, Ali, David Chan, and Mohammad H. Mahoor, "Going deeper in facial expression recognition using deep neural networks." Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016.
3 Jung, Heechul, et al. "Joint fine-tuning in deep neural networks for facial expression recognition." Proceedings of the IEEE International Conference on Computer Vision, 2015.
4 Lopes, Andre Teixeira, Edilson de Aguiar, and Thiago Oliveira-Santos, "A facial expression recognition system using convolutional networks," Graphics, Patterns and Images (SIBGRAPI), 2015 28th SIBGRAPI Conference on, IEEE, 2015.
5 Hamester, Dennis, Pablo Barros, and Stefan Wermter. "Face expression recognition with a 2-channel convolutional neural network," Neural Networks (IJCNN), 2015 International Joint Conference on. IEEE, 2015.
6 W. Bainbridge, P. Isola, and A. Oliva, "The intrinsic memorability of face photographs," Journal of Experimental Psychology: General, 142(4):1323-1334, 2013.   DOI
7 S. Setty and et al, "Indian Movie Face Database: A Benchmark for FaceRecognition Under Wide Variation," In NCVPRIPG, 2013.
8 P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression," in Proceedings of the IEEE Workshop on CVPR for Human Communicative Behavior Analysis, 2010.
9 Ma, Correll, and Wittenbrink, The Chicago Face Database: A Free Stimulus Set of Faces and Norming Data, Behavior Research Methods, 47, 1122-1135.   DOI
10 ESRC 3D Face Database, http://pics.stir.ac.uk/ESRC/
11 J. Van der Schalk, S. T. Hawk, A. H. Fischer, and B. J. Doosje, Moving faces, looking places: The Amsterdam Dynamic Facial Expressions Set (ADFES), Emotion, 11, 907-920. DOI: 10.1037/a0023853, 2011.   DOI
12 D. Lundqvist, A. Flykt, and A.Ohman (1998), The Karolinska Directed Emotional Faces - KDEF, CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, ISBN 91-630-7164-9.
13 H. O'Reilly, D. Pigat, S. Fridenson, S. Berggren, S. Tal, O. Golan, S. B"olte, S. Baron-Cohen and D. Lundqvist, The EU-Emotion Stimulus Set: A Validation Study, Behavior Research Methods. DOI: 10.3758/s13428-015-0601-4, 2015.   DOI
14 Viola and Jones, "Rapid object detection using a boosted cascade of simple features," Computer Vision and Pattern Recognition, 2001.
15 M. Olszanowski, G. Pochwatko, K. Kuklinski, M. Scibor-Rylski, P. Lewinski and RK. Ohme, Warsaw Set of Emotional Facial Expression Pictures: A validation study of facial display photographs, Front. Psychol, 5:1516. doi: 10.3389/fpsyg.2014.01516, 2015.   DOI
16 A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, 2012.
17 Learn facial expressions from an image, https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognitionchallenge/data
18 M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," In European Conference on Computer Vision , Springer International Publishing, pp. 818-833, September 2014.
19 K. Simonyan, and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," In Proc. International Conference on Learning Representations, http://arxiv.org/abs/1409.1556 (2014).
20 P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus and Y. LeCun, "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks." In Proc. ICLR, 2014.
21 P. Burkert, F. Trier, M. Z. Afzal, A. Dengel, and M. Liwicki. Dexpression: "Deep convolutional neural network for expression recognition," .CoRR, abs/1509.05371, 2015.