Journal of Korea Game Society (한국게임학회 논문지)

Korea Game Society (한국게임학회)
권호 표지
DOI QR코드

DOI QR Code

A GAN-based face rotation technique using 3D face model for game characters

3D 얼굴 모델 기반의 GAN을 이용한 게임 캐릭터 회전 기법

  • Kim, Handong (Dept. of Computer Science, Graduate school, Sangmyung Univ.) ;
  • Han, Jongdae (Dept. of Computer Science, Sangmyung Univ.) ;
  • Yang, Heekyung (Dept. of SW Convergence, Sangmyung Univ.) ;
  • Min, Kyungha (Dept. of Computer Science, Sangmyung Univ.)
  • 김한동 (상명대학교 일반대학원 컴퓨터학과) ;
  • 한종대 (상명대학교 컴퓨터과학과) ;
  • 양희경 (상명대학교 SW융합학부) ;
  • 민경하 (상명대학교 컴퓨터과학과)
  • Received : 2021.03.10
  • Accepted : 2021.05.01
  • Published : 2021.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

This paper shows the face rotation applicable to game character facial illustration. Existing studies limited data to human face data, required a large amount of data, and the synthesized results were not good. In this paper, the following method was introduced to solve the existing problems of existing studies. First, a 3D model with features of the input image was rotated and then rendered as a 2D image to construct a data set. Second, by designing GAN that can learn features of various poses from the data built through the 3D model, the input image can be synthesized at a desired pose. This paper presents the results of synthesizing the game character face illustration. From the synthesized result, it can be confirmed that the proposed method works well.

본 논문은 게임 캐릭터 얼굴 일러스트레이션에 적용할 수 있는 안면 회전 기술(Face rotation) 기술을 제안한다. 기존의 진행된 연구들은 실제 사람의 얼굴 데이터에 대해서로 데이터를 한정하였으며 방대한 양의 데이터를 필요로 하였고 합성된 결과물이 좋지 못한 문제가 있었다. 본 논문에서는 기존 연구들의 존재하는 문제를 해결하기 위해 다음과 같은 방법을 도입하였다. 첫째, 입력 이미지가 갖고 있는 특징을 입힌 3D 모델을 회전시키고 다시 2D 이미지로 렌더링하여 학습 및 평가에 필요한 데이터 셋을 구축하였다. 둘째, 3D 모델을 통해 구축된 데이터에서 다양한 각도의 특징을 학습할 수 있는 적대적 생성 모델(Generative Adversarial Networks)을 설계하여 입력된 이미지를 원하는 각도로 합성할 수 있다. 논문에서는 실제 게임 캐릭터 얼굴 일러스트레이션 합성 결과를 제시한다. 합성 결과를 통해 논문에서 제안하는 방법이 잘 동작함을 확인할 수 있다.

Keywords

References

  1. I.Goodfellow, J.Pouget-Abadie, M.Mirza, B.Xu, D.Warde-Farley, S.Ozair, A.Courville, Y.Bengio. "Generative adversarial networks.". Advances in Neural Information Processing Systems 27, 2014.
  2. T.Hassner, S.Harel, E.Paz, R.Enbar. "Effective face frontalization in unconstrained images." Proceedings of the IEEE conference on computer vision and pattern recognition, pp4295-4303, 2015.
  3. Z.Xiangyu, L.Zhen, Y.Junjie, Y.Dong, L.Stan Z. "High-fidelity pose and expression normalization for face recognition in the wild." Proceedings of the IEEE conference on computer vision and pattern recognition, pp787-796, 2015.
  4. J.Moniz, C.Beckham, S.Rajotte, S.Homari, C.Pal. "Unsupervised depth estimation, 3d face rotation and replacement." arXiv preprint arXiv:1803.09202, 2018.
  5. T.Luan, Y.Xi, L.Xiaoming. "Disentangled represerntation learning gan for pose-invariant face recognition." Proceedings of the IEEE conference on computer vision and pattern recognition, pp1415-1424, 2017
  6. H.Rui, Z.Shu, L.Tianyu, H.Ran. "Beyond face rotation: Global and local perception gan for photorealistic and identity preserving frontal view synthesis." Proceedings of the IEEE international conference on computer vision, pp2439-2448, 2017
  7. H.Yibo, W.Xiang, Y.Bing, H.Ran, S.Zhenan. "Pose-guided photorealistic face rotation." Proceedings of the IEEE conference on computer vision and pattern recognition, pp8398-8406, 2018
  8. Q.Yichen, D.Weihong, H.Jiani. "Unsupervised face normalization with extreme pose and expression in the wild." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp9851-9858, 2019.
  9. Y.Xi, Y.Xiang, S.Kihyuk, L.Xiaoming, C.Manmohan. "Towards large-pose face frontalization in the wild." Proceedings of the IEEE international conference on computer vision, pp3990-3999, 2017
  10. D.Jiankang, C.Shiyang, X.Niannan, Z.Yuxiang, Z.Stefanos. "Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition." Proceedings of the IEEE conference on computer vision and pattern recognition, pp7093-7102, 2018.
  11. C.Jie, H.Yibo, Z.Hongwen, H.Ran, S.Zheman. "Learning a high fidelity pose invariant model for high-resolution face frontalization." arXiv preprint arXiv:1806.08472, 2018.
  12. Z.Hang, L.Jihao, L.Ziwei, L.Yu, W.Xiaogang. "Rotate-and-render: Unsupervised photo-realistic face rotation from single-view images." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp5911-5920, 2020.
  13. Z.Jun-Yan, P.Taesung, I.Phillip, E.Alexei A. "Unpaired image-to-image translation using cycle-consistent adversarial networks." Proceedings of the IEEE International conference on computer vision, pp2223-2232, 2017.
  14. J.Guo, X.Zhu, Z.Lei. "3DDFA". https://github.com/cleardusk/3DDFA, 2018
  15. Guardian tales, Kong studois, 2020. Phoenix Wright: Ace Attorney, CAPCOM, 2001. The war of Genesis, Softmax, 1995. Hearthstone, Blizzard, 2014.
  16. PyTorch, 2016, https://pytorch.org/
  17. Yi, D., Lei, Z., Liao, S., Li, S. Z. "Learning face representation from scratch." arXiv preprint arXiv:1411.7923, 2014.
  18. Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J. "Ms-celeb-1m: A dataset and benchmark for large-scale face recognition." In European conference on computer vision. pp87-102. 2016.