Browse > Article
http://dx.doi.org/10.9708/jksci.2022.27.09.041

Facial Expression Recognition through Self-supervised Learning for Predicting Face Image Sequence  

Yoon, Yeo-Chan (Dept. of Artificial Intelligence, Jeju National University)
Kim, Soo Kyun (Dept. of Computer Engineering, Jeju National University)
Publication Information
Journal of the Korea Society of Computer and Information / v.27, no.9, 2022 , pp. 41-47 More about this Journal
Abstract
In this paper, we propose a new and simple self-supervised learning method that predicts the middle image of a face image sequence for automatic expression recognition. Automatic facial expression recognition can achieve high performance through deep learning methods, however, generally requires a expensive large data set. The size of the data set and the performance of the algorithm are tend to be proportional. The proposed method learns latent deep representation of a face through self-supervised learning using an existing dataset without constructing an additional dataset. Then it transfers the learned parameter to new facial expression reorganization model for improving the performance of automatic expression recognition. The proposed method showed high performance improvement for two datasets, CK+ and AFEW 8.0, and showed that the proposed method can achieve a great effect.
Keywords
Facial Expression Recognition; Unsupervised-learning; Self-supervised Learning; Deep Learning; Artificial Intelligence;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. C. Yoon, "Can We Exploit All Datasets? Multimodal Emotion Recognition Using Cross-Modal Translation," in IEEE Access, vol. 10, pp. 64516-64524, 2022, doi: 10.1109/ACCESS.2022.3183587.   DOI
2 Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Conference on Computer Vision, pp. 649-666. Springer, 2016a.
3 Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1422-1430, 2015.
4 Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision, pp. 69-84. Springer, 2016.
5 Li, Chun-Liang, et al. "Cutpaste: Self-supervised learning for anomaly detection and localization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
6 P. Ekman and W. V. Friesen, "Constants across cultures in the face and emotion." Journal of personality and social psychology, vol. 17, no. 2, pp. 124-129, 1971.   DOI
7 Meng, Debin, et al. "Frame attention networks for facial expression recognition in videos." 2019 IEEE international conference on image processing (ICIP). IEEE, 2019.
8 Li, Shan, and Weihong Deng. "Deep facial expression recognition: A survey." IEEE transactions on affective computing (2020).
9 Weiss, Karl, Taghi M. Khoshgoftaar, and DingDing Wang. "A survey of transfer learning." Journal of Big data 3.1 (2016): 1-40.   DOI
10 Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for automatic colorization. In European Conference on Computer Vision, pp. 577-593. Springer, 2016.
11 Gidaris, Spyros, Praveer Singh, and Nikos Komodakis. "Unsupervised representation learning by predicting image rotations." arXiv preprint arXiv:1803.07728 (2018).
12 Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017. 2, 5
13 Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In AAAI, 2020. 5, 12
14 Bergmann, Paul, et al. "The MVTec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection." International Journal of Computer Vision 129.4 (2021): 1038-1059.   DOI
15 Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews, "The extended cohnkanade dataset (ck+): A complete dataset for action unit and emotion-specified expression," in CVPRW, 2010.
16 Zhou, Tinghui, et al. "Unsupervised learning of depth and ego-motion from video." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
17 A. Mollahosseini, D. Chan, and M. H. Mahoor, "Going deeper in facial expression recognition using deep neural networks," in Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016, pp. 1-10.
18 Abhinav Dhall, Amanjot Kaur, Roland Goecke, and Tom Gedeon, "Emotiw 2018: Audio-video, student engagement and group-level affect prediction," arXiv preprint:1808.07773, 2018