Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.3.369

Facial Image Synthesis by Controlling Skin Microelements  

Kim, Yujin (Inha University, Department of Electrical & Computer Engineering)
Park, In Kyu (Inha University, Department of Electrical & Computer Engineering)
Publication Information
Journal of Broadcast Engineering / v.27, no.3, 2022 , pp. 369-377 More about this Journal
Abstract
Recent deep learning-based face synthesis research shows the result of generating a realistic face including overall style or elements such as hair, glasses, and makeup. However, previous methods cannot create a face at a very detailed level, such as the microstructure of the skin. In this paper, to overcome this limitation, we propose a technique for synthesizing a more realistic facial image from a single face label image by controlling the types and intensity of skin microelements. The proposed technique uses Pix2PixHD, an Image-to-Image Translation method, to convert a label image showing the facial region and skin elements such as wrinkles, pores, and redness to create a facial image with added microelements. Experimental results show that it is possible to create various realistic face images reflecting fine skin elements corresponding to this by generating various label images with adjusted skin element regions.
Keywords
Face synthesis; Image-to-Image Translation; Skin elements;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. -Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," Proc. IEEE International Conference on Computer Vision, October 2017. doi: https://doi.org/10.1109/iccv.2017.244   DOI
2 Mark-Vu, https://www.psiplus.co.kr/page/?M2_IDX=8160 (accessed Mar. 15, 2022).
3 I. J. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Proc. Advances in Neural Information Processing Systems, December 2014.
4 M. Mirza and S. Osindero, "Conditional generative adversarial nets," arXiv preprint arXiv:1411.1784, 2014.
5 Y. Choi, M. Choi, M. Kim, J. -W. Ha, S. Kim, and J. Choo, "StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation," Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2018. doi: https://doi.org/10.1109/cvpr.2018.00916   DOI
6 Y. Shen, J. Gu, X. Tang, and B. Zhou, "Interpreting the latent space of GANs for semantic face editing," Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020. doi: https://doi.org/10.1109/cvpr42600.2020.00926   DOI
7 T. -C. Wang, M. -Y. Liu, J. -Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, "High-resolution image synthesis and semantic manipulation with conditional GANs," Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2018. doi: https://doi.org/10.1109/cvpr.2018.00917   DOI
8 E. Richardson, Y. Alaluf, O. Patashnik, Y. Nitzan, Y. Azar, S. Shapiro, and D. Cohen-Or, "Encoding in style: A StyleGAN encoder for image-to-image translation," Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2021. doi: https://doi.org/10.1109/cvpr46437.2021.00232   DOI
9 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, Vol. 86, No. 11, pp. 2278-2324, November 1998. doi: https://doi.org/10.1109/5.726791   DOI
10 Z. Huang, S. Chen, J. Zhang, and H. Shan, "PFA-GAN: Progressive face aging with generative adversarial network," IEEE Trans. on Information Forensics and Security, Vol. 16, pp. 2031-2045, December 2020. doi: https://doi.org/10.1109/tifs.2020.3047753   DOI
11 P. Isola, J. -Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2017. doi: https://doi.org/10.1109/cvpr.2017.632   DOI
12 U. Demir and G. Unal, "Patch-based image inpainting with generative adversarial networks," arXiv preprint arXiv:1803.07422, 2018.
13 T. Karras, T. Alia, S. Laine, and J. Lehtinen, "Progressive growing of GANs for improved quality, stability, and variation," Proc. International Conference on Learning Representations, April 2018.
14 T. Karras, S. Lanine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, "Analyzing and improving the image quality of StyleGAN," Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020. doi: https://doi.org/10.1109/cvpr42600.2020.00813   DOI
15 T. Karras, M. Aittala, J. Hellsten, S. Lanine, J. Lehtinen, and T. Aila, "Training generative adversarial networks with limited data," Proc. Advances in Neural Information Processing Systems, December 2020.
16 G. Perarnau, J. van de Weijer, B. Raducanu, and J. M. Alvarez, "Invertible conditional GANs for image editing," arXiv preprint arXiv:1611.06355, 2016.
17 C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang, "Bisenet: Bilateral segmentation network for real-time semantic segmentation," Proc. European Conference on Computer Vision, September 2018. doi: https://doi.org/10.1007/978-3-030-01261-8_20   DOI
18 Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," Proc. IEEE International Conference on Computer Vision, December 2015. doi: https://doi.org/10.1109/iccv.2015.425   DOI
19 Z. He, W. Zuo, M. Kan, S. Shan, and X. Chen, "AttGAN: Facial attribute editing by only changing what you want," IEEE Trans. on Image Processing, Vol. 28, No. 11, pp. 5464-5478, May 2019. doi: https://doi.org/10.1109/tip.2019.2916751   DOI
20 C. -H. Lee, Z. Liu, L. Wu, and P. Luo, "MaskGAN: Towards diverse and interactive facial image manipulation," Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020. doi: https://doi.org/10.1109/cvpr42600.2020.00559   DOI