Browse > Article
http://dx.doi.org/10.9708/jksci.2020.25.05.001

Improved STGAN for Facial Attribute Editing by Utilizing Mask Information  

Yang, Hyeon Seok (Dept. of Computer Science and Engineering, Hanyang University)
Han, Jeong Hoon (Dept. of Computer Science and Engineering, Hanyang University)
Moon, Young Shik (Dept. of Computer Science and Engineering, Hanyang University)
Publication Information
Journal of the Korea Society of Computer and Information / v.25, no.5, 2020 , pp. 1-9 More about this Journal
Abstract
In this paper, we propose a model that performs more natural facial attribute editing by utilizing mask information in the hair and hat region. STGAN, one of state-of-the-art research of facial attribute editing, has shown results of naturally editing multiple facial attributes. However, editing hair-related attributes can produce unnatural results. The key idea of the proposed method is to additionally utilize information on the face regions that was lacking in the existing model. To do this, we apply three ideas. First, hair information is supplemented by adding hair ratio attributes through masks. Second, unnecessary changes in the image are suppressed by adding cycle consistency loss. Third, a hat segmentation network is added to prevent hat region distortion. Through qualitative evaluation, the effectiveness of the proposed method is evaluated and analyzed. The method proposed in the experimental results generated hair and face regions more naturally and successfully prevented the distortion of the hat region.
Keywords
Facial attribute editing; GAN; Deep learning; Mask; STGAN;
Citations & Related Records
연도 인용수 순위
  • Reference
1 I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," Advances in neural information processing systems, pp. 2672-2680, Dec. 2014.
2 M. Mirza and S. Osindero, "Conditional Generative Adversarial Nets," arXiv preprint arXiv:1411.1784, pp. 1-7, Nov. 2014.
3 G. Perarnau, J. V. D. Weijer, B. Raduanu, and J. M. Alvarez, "Invertible Conditional GANs for Image Editing," NIPS 2016 Workshop on Adversarial Training, pp. 1-9, Dec. 2016.
4 A. B. L. Larsen, S. K. Sonderby, H. Larochelle, and O. Winther, "Autoencoding Beyond Pixels using a Learned Similarity Metric," Proceedings of the 33rd International Conference on Machine Learning, Vol. 48, pp. 1-9, Feb. 2016.
5 Z. He, W. Zuo, and S. Shan, "AttGAN: Facial Attribute Editing by Only Changing What You Want." IEEE Transactions on Image Processing, Vol. 28, No. 11, pp. 5464-5478, May. 2019. DOI: 10.1109/TIP.2019.2916751   DOI
6 M. Liu, Y. Ding, M. Xia, X. Liu, E. Ding, W. Zuo, and S. Wen, "STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3673-3682, Jun. 2019.
7 W. Shen and R. Liu, "Learning Residual Images for Face Attribute Manipulation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4030-4038, Jul. 2017. DOI: 10.1109/CVPR.2017.135
8 G. Zhang, M. Kan, S. Shan, and X. Chen, "Generative Adversarial Network with Spatial Attention for Face Attribute Editing," Proceedings of the European Conference on Computer Vision (ECCV), pp. 417-432, Sep. 2018. DOI: 10.1007/978-3-030-01231-1_26
9 P. Chen, Q. Xiao, J. Xu, X. Dong, and L. Sun, "Facial Attribute Editing using Semantic Segmentation," 2019 International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS), pp. 97-103, May 2019. DOI: 10.1109/HPBDIS.2019.8735455
10 H. S. Yang and Y. S. Moon, "Face Attribute Editing using AttGAN and Guide Mask," 2019 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1-3, Jan. 2019. DOI: 10.23919/ELINFOCOM.2019. 8706471
11 X. Zheng, Y. Guo, H. Huang, Y. Li, and R. He, "A Survey to Deep Facial Attribute Analysis," International Journal of Computer Vision, pp. 1-33, Mar. 2020. DOI: 10.1007/s11263-0 20-01308-z
12 J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks," Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, Oct. 2017. DOI: 10.1109/ICCV.2017.244
13 P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efos, "Image-to-Image Translation with Conditional Adversarial Networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, Jul. 2017. DOI: 10.1109/CVPR.2017.632
14 Y. Choi, M. Choi, M. Kim, J. W. Ha, S. Kim, and J. Choo, "StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789-8797, Jun. 2018. DOI: 10.1109/CVPR.2018.00916
15 Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep Learning Face Attributes in the Wild," Proceedings of the IEEE International Conference on Computer Vision, pp. 3730-3738, Dec. 2015. DOI: 10.1109/ICCV.2015.425
16 H. S. Yang, J. H. Han, Y. C. Cho, H. G. Lee, Y. Park, and Y. S. Moon, "Study on Performance Improvement of SAGAN using Mask," Proceeding of 2019 Korea Signal Processing Conference, pp. 2557-2560, Sep. 2019.
17 C. H. Lee, Z. Liu, L. Wu, and P. Luo, "MaskGAN: Towards Diverse and Interactive Facial Image Manipulation," arXiv preprint arXiv:1907.11922v2, pp. 1-20, Apr. 2020.