DOI QR코드

DOI QR Code

3D Human Shape Deformation using Deep Learning

딥러닝을 이용한 3차원 사람모델형상 변형

  • Received : 2020.02.01
  • Accepted : 2020.03.30
  • Published : 2020.04.30

Abstract

Recently, rapid and accurate 3D models creation is required in various applications using virtual reality and augmented reality technology. In this paper, we propose an on-site learning based shape deformation method which transforms the clothed 3D human model into the shape of an input point cloud. The proposed algorithm consists of two main parts: one is pre-learning and the other is on-site learning. Each learning consists of encoder, template transformation and decoder network. The proposed network is learned by unsupervised method, which uses the Chamfer distance between the input point cloud form and the template vertices as the loss function. By performing on-site learning on the input point clouds during the inference process, the high accuracy of the inference results can be obtained and presented through experiments.

최근 가상현실 및 증강 현실 기술을 이용한 다양한 응용분야가 각광받으면서 빠르고 정확한 3차원 모델 생성이 요구되고 있다. 본 논문에서는 옷을 입은 3차원 사람 모델을 포인트 클라우드의 형상으로 변형하는 온-사이트 학습 (On-site learning) 기반 형상 변형 방법을 제안한다. 제안하는 알고리즘은 사전 학습과 온-사이트 학습 두 개의 파트로 구성되어 있으며, 각각의 학습은 인코더 네트워크, 템플릿 변형 네트워크, 디코더 네트워크로 구성된다. 딥러닝 네트워크 학습은 3차원 포인트 클라우드와 템플릿 정점 사이의 챔퍼 거리 (Chamfer distance)를 주요 손실 함수로 사용하는 비지도 학습을 적용한다. 입력된 포인트 클라우드 형태의 데이터에 대해 온-사이트 학습을 진행함으로써 추론의 결과물에 대한 높은 정확도를 얻을 수 있으며 이를 실험을 통해 제시한다.

Keywords

References

  1. Allen, B., Curless, B., Popovic, Z. (2002). Articulated Body Deformation from Range Scan Data, Transactions On Graphics, 21(3), 612-619. https://doi.org/10.1145/566654.566626
  2. Allen, B., Curless, B., Popovic, Z. (2003). The Space of Human Body Shapes: Reconstruction and Parameterization from Range Scans, Transactions On Graphics, 22(3), 587-594. https://doi.org/10.1145/882262.882311
  3. Chen, X., Pang, A., Zhu, Y., Li, Y., Luo, X., Zhang, G., Wang, P., Zhang, Y., Li, S., Yu, J. (2019). Towards 3D Human Shape Recovery Under Clothing, arXiv preprint arXiv:1904.02601.
  4. Daz Productions, https://www.daz3d.com/(accessed Aug 10, 2019)
  5. Fan, H., Su, H., Guibas, L. (2017). A Point Set Generation Network for 3D Object Reconstruction from a Single Image, The IEEE Conference on Computer Vision and Pattern Recognition, July. 21-26, Honolulu, USA, pp. 605-613.
  6. Groueix, T., Fisher, M., Kim, Vladimir, G., Russell, Bryan, C., Aubry, M. (2018). 3D-CODED: 3D Correspondences by Deep Deformation, The European Conference on Computer Vision, Sep. 8-14, Munich, Germany, pp. 230-246.
  7. Jaderberg, M., Simonyan, K., Zisserman, A. (2015). Spatial Transformer Networks, The Advances in Neural Information Processing Systems, Dec. 7-12, Montreal, Canada, pp. 2017-2025.
  8. Loper, M., Mahmood, N., Romero, J., Moll, G., Black, MJ. (2015) SMPL: A Skinned Multi-person Linear Model, The ACM Transactions On Graphics, 34(6), 248.
  9. Qi, CR., Su, H., Mo, K., Guibas, LJ. (2017). Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation, The IEEE Conference on Computer Vision and Pattern Recognition, July. 21-26, Honolulu, USA, pp. 652-660.
  10. Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., Jiang, Yu-Gang. (2018). Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images, The European Conference on Computer Vision, Sep. 8-14, Munich, Germany, pp. 52-67.
  11. Zuffi, S., Black, MJ. (2015). The Stitched Puppet: A Graphical Model of 3D Human Shape and Pose, The IEEE Conference on Computer Vision and Pattern Recognition, June. 7-12, Boston, USA, pp. 3537-3546.

Cited by

  1. 마이크로스코프 이미지의 딥러닝 기반 이상검출 vol.26, pp.2, 2021, https://doi.org/10.9723/jksiis.2021.26.2.001