Browse > Article
http://dx.doi.org/10.33851/JMIS.2021.8.2.85

Frontal Face Generation Algorithm from Multi-view Images Based on Generative Adversarial Network  

Heo, Young- Jin (Dept. of IT Engineering, Sookmyung Women's University)
Kim, Byung-Gyu (Dept. of IT Engineering, Sookmyung Women's University)
Roy, Partha Pratim (IIT Roorkee)
Publication Information
Journal of Multimedia Information System / v.8, no.2, 2021 , pp. 85-92 More about this Journal
Abstract
In a face, there is much information of person's identity. Because of this property, various tasks such as expression recognition, identity recognition and deepfake have been actively conducted. Most of them use the exact frontal view of the given face. However, various directions of the face can be observed rather than the exact frontal image in real situation. The profile (side view) lacks information when comparing with the frontal view image. Therefore, if we can generate the frontal face from other directions, we can obtain more information on the given face. In this paper, we propose a combined style model based the conditional generative adversarial network (cGAN) for generating the frontal face from multi-view images that consist of characteristics that not only includes the style around the face (hair and beard) but also detailed areas (eye, nose, and mouth).
Keywords
GAN; StyleGAN; cGAN; Deep learning; Classification; Frontal face;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Maja Pantic and Ioannis Patras, "Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 36, no. 2, pp. 433-449, 2006.   DOI
2 Diederik P Kingma and Max Welling, "Auto-encoding variational bayes," arXiv preprint arXiv:1312.6114, 2013.
3 Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei AEfros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," in Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
4 Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim, "Learning to discover cross-domain relations with generative adversarial networks," arXiv preprint arXiv:1703.05192, 2017.
5 The FEI face database of Laboratory of FEI, "http://fei.edu.br/cet/facedatabase.html".
6 Kaiming He, Xiangyu Zhang, Shaoqing Ren, and JianSun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
7 Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, "Generative adversarial nets," in Proceeding of Advances in neural information processing systems, pp. 2672-2680, 2014.
8 Mehdi Mirza and Simon Osindero, "Conditional generative adversarial nets," arXiv preprint arXiv: 1411.1784, 2014.
9 Alec Radford, Luke Metz, and Soumith Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv: 1511.06434, 2015.
10 Ji-Hae Kim, Byung-Gyu Kim, Partha Pratim Roy, and Da-Mi Jeong, "Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure," IEEE Access, vol. 7, pp. 41273-41285, 2019.   DOI
11 CIFAR10 dataset of Laboratory of Toronto, "https://www.cs.toronto.edu/kriz/cifar.html".
12 Rohit Srivastava, Ravi Tomar, Ashutosh Sharma, Gaurav Dhiman, Naveen Chilamkurti, and Byung-Gyu Kim, "Real-Time Multimodal Biometric Authentication of Human Using Face Feature Analysis," Computers, Materials & Continua, vol. 49, no.1, pp. 1-19 (DOI:10.32604/cmc.2021.015466), 2021.   DOI
13 Ji-Hae Kim, Gwang-Soo Hong, Byung-Gyu Kim, and Debi P. Dogra, "deepGesture: Deep learning-based gesture recognition scheme using motion sensors," Displays, vol. 55, pp. 34-45 (https://doi.org/10.1016/j.displa.2018.08.001), 2018.   DOI
14 Dong-hyeon Kim, Dong-seok Lee, and Soon-kak Kwon, "Fall Situation Recognition by Body Centerline Detection using Deep Learning," Journal of Multimedia Information System, vol. 7, no. 4, pp. 257-262, 2020.   DOI
15 Woon-Ha Yeo, Young-Jin Heo, Young-Ju Choi, and Byung-Gyu Kim, "Place Classification Algorithm Based on Semantic Segmented Objects," Applied Sciences, vol. 2020, no. 10, p. 9069 (https://doi.org/10.3390/app10249069), Dec. 2020.   DOI
16 S. Mukherjee, S. Ghosh, S. Ghosh, P. Kumar, and P. P. Roy, "Predicting Video-frames Using Encoder-convlstm Combination," in Proceeding of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2027-2031 (doi: 10.1109/ICASSP.2019.8682158), 2019.   DOI
17 Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," arXiv preprint arXiv:1602.07261, 2016.
18 Yunjey Choi, Minje Choi, Munyoung Kim, JungWooHa, Sunghun Kim, and Jaegul Choo, "Stargan: Unified generative adversarial networks for multi-domainimage-to-image translation," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789-8797, 2018.
19 Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Proceeding of 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, 2009.
20 Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou, "Interfacegan: Interpreting the disentangled face representation learned by gans," arXiv preprint arXiv:2005.09635, 2020.
21 Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao, "Joint face detection and alignment using multitask cascaded convolutional networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.   DOI
22 Dami Jeong, Byung-Gyu Kim, and Suh-Yeon Dong, "Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition," Sensors, vol. 2020, no. 20, p. 1963 (https://doi.org/10.3390/s20071936), 2020.   DOI
23 Tero Karras, Samuli Laine, and Timo Aila, "A style-based generator architecture for generative adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp.4401-4410, 2019.
24 Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby, "Big transfer (bit): General visual representation learning," arXiv preprint arXiv:1912. 11370, vol. 6, 2019.