Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.5.619

Multi-attribute Face Editing using Facial Masks  

Ambardi, Laudwika (Department of Electrical and Computer Engineering)
Park, In Kyu (Department of Electrical and Computer Engineering)
Hong, Sungeun (Department of Electrical and Computer Engineering)
Publication Information
Journal of Broadcast Engineering / v.27, no.5, 2022 , pp. 619-628 More about this Journal
Abstract
Although face recognition and face generation have been growing in popularity, the privacy issues of using facial images in the wild have been a concurrent topic. In this paper, we propose a face editing network that can reduce privacy issues by generating face images with various properties from a small number of real face images and facial mask information. Unlike the existing methods of learning face attributes using a lot of real face images, the proposed method generates new facial images using a facial segmentation mask and texture images from five parts as styles. The images are then trained with our network to learn the styles and locations of each reference image. Once the proposed framework is trained, we can generate various face images using only a small number of real face images and segmentation information. In our extensive experiments, we show that the proposed method can not only generate new faces, but also localize facial attribute editing, despite using very few real face images.
Keywords
Face editing; Image synthesis; Facial mask; Data privacy;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Wang, Ting-Chun, Arun Mallya, and Ming-Yu Liu. "One-shot free-view neural talking-head synthesis for video conferencing," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10039-10049, 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00991   DOI
2 Vitadhani, Agastya, Kalamullah Ramli, and Prima Dewi Purnamasari. "Detection of Clickbait Thumbnails on YouTube Using TesseractOCR, Face Recognition, and Text Alteration," Proceedings of International Conference on Artificial Intelligence and Computer Science Technology (ICAICST), pp. 56-61, 2021. doi: https://doi.org/10.1109/ICAICST53116.2021.9497811   DOI
3 Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784, 2014. doi: https://doi.org/10.48550/arXiv.1411.1784   DOI
4 Isola, Phillip, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125-1134, 2017. doi: https://doi.org/10.1109/CVPR.2017.632   DOI
5 Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2414-2423, 2016. doi:https://doi.org/10.1109/CVPR.2016.265   DOI
6 Grassucci, Eleonora, Edoardo Cicero, and Danilo Comminiello. "Quaternion generative adversarial networks," In Generative Adversarial Learning: Architectures and Applications, pp. 57-86. Springer, 2022. doi: https://doi.org/10.1007/978-3-030-91390-8_4   DOI
7 Korshunova, Iryna, Wenzhe Shi, Joni Dambre, and Lucas Theis. "Fast face-swap using convolutional neural networks," Proceedings of the IEEE International Conference on Computer Vision, pp. 3677-3685, 2017. doi: https://doi.org/10.1109/ICCV.2017.397   DOI
8 Huang, Xun, and Serge Belongie. "Arbitrary style transfer in real-time with adaptive instance normalization," Proceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017. doi: https://doi.org/10.48550/arXiv.1703.06868   DOI
9 Wang, X. and Gupta, A., "Generative image modeling using style and structure adversarial networks," Proceedings of European Conference on Computer Vision, pp. 318-335, 2016. doi: https://doi.org/10.1007/978-3-319-10578-9_24   DOI
10 Zhu, Yuhao, Qi Li, Jian Wang, Cheng-Zhong Xu, and Zhenan Sun. "One shot face swapping on megapixels," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4834-4844, 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00480   DOI
11 Gu, Shuyang, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen, and Lu Yuan. "Mask-guided portrait editing with conditional gans," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3436-3445, 2019. doi: https://doi.org/10.1109/CVPR.2019.00355   DOI
12 Zhu, Peihao, Rameen Abdal, Yipeng Qin, and Peter Wonka. "Sean: Image synthesis with semantic region-adaptive normalization," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5104-5113, 2020. doi: https://doi.org/10.48550/arXiv.1911.12861   DOI
13 Karras, Tero, Timo Aila, Samuli Laine, and Jaakko Lehtinen. "Progressive Growing of GANs for Improved Quality, Stability, and Variation," Proceedings of International Conference on Learning Representations, 2018. doi: https://doi.org/10.48550/arXiv.1710.10196   DOI
14 Kim, Hyunsu, Yunjey Choi, Junho Kim, Sungjoo Yoo, and Youngjung Uh. "Exploiting spatial dimensions of latent in gan for real-time image editing," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 852-861, 2021. doi: https://doi.org/10.48550/arXiv.2104.14754   DOI
15 Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. "Generative adversarial nets," Proceedings of Advances in Neural Information Processing Systems, 2014. doi: https://doi.org/10.48550/arXiv.1406.2661   DOI
16 Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434, 2015. doi: https://doi.org/10.48550/arXiv.1511.06434   DOI
17 Choi, Yunjey, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8789-8797, 2018. doi: https://doi.org/10.1109/CVPR.2018.00916   DOI
18 Xu, Yangyang, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, and Shengfeng He. "High-resolution face swapping via latent semantics disentanglement," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7642-7651, 2022. doi: https://doi.org/10.48550/arXiv.2203.15958   DOI
19 Zhou, Hang, Jihao Liu, Ziwei Liu, Yu Liu, and Xiaogang Wang. "Rotate-and-render: Unsupervised photorealistic face rotation from single-view images," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5911-5920, 2020. doi: https://doi.org/10.48550/arXiv.2003.08124   DOI
20 Wang, Ting-Chun, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. "High-resolution image synthesis and semantic manipulation with conditional gans," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8798-8807, 2018. doi:https://doi.org/10.1109/CVPR.2018.00917   DOI