Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Fashion-show Animation Generation using a Single Image to 3D Human Reconstruction Technique

이미지에서 3차원 인물복원 기법을 사용한 패션쇼 애니메이션 생성기법

  • Received : 2019.08.06
  • Accepted : 2019.09.09
  • Published : 2019.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we introduce the technology to convert a single human image into a fashion show animation video clip. The technology can help the customers confirm the dynamic fitting result when combined with the virtual try on technique as well as the interesting experience to a normal person of being a fashion model. We developed an extended technique of full human 2D to 3D inverse modeling based on SMPLify human body inverse modeling technique, and a rigged model animation method. The 3D shape deformation of the full human from the body model was performed by 2 part deformation in the image domain and reconstruction using the estimated depth information. The quality of resultant animation videos are made to be publically available for evaluation. We consider it is a promising approach for commercial application when supplemented with the post - processing technology such as image segmentation technique, mapping technique and restoration technique of obscured area.

본 논문은 단일 이미지를 패션쇼 워킹 영상으로 변환하는 기술을 소개한다. 일반인이 가상으로 패션모델이 되어 보는 흥미로운 응용일 뿐 아니라, 나아가 가상 착용기술과 함께 결합하게 되면 의상착용결과의 동적인 확인이 가능한 기술이다. 본 논문에서 사용한 기술은 이미지에서 3차원 인간신체 모델을 추정 복원해 주는 SMPLify 기법에 기초하여, 인체 모델에서 의상을 포함한 사람으로 모델을 확장하고, 이에 애니메이션 기법을 적용하여 구현되었다. 인체와 의상을 포한한 사람의 3차원 모델은 2차원 이미지 상에서 기하변형과 깊이정보를 사용하여 복원하였다. 패션 데이터 셋에 적용해 본 결과 정자세의 경우에는 성공적인 수준의 결과를 보였으나, 상용수준의 성능을 위해서는 이미지의 분할 기술, 매핑기술 및 가려진 영역의 복원기술 등 선 후처리 기술에 보완이 필요한 것으로 확인되었다.

Keywords

References

  1. Ahn H., (2018a). Online Virtual Try On using Mannequin Cloth Pictures, Journal of the Korea Industrial Information Systems Research, 23(6), 29-38. https://doi.org/10.9723/JKSIIS.2018.23.6.029
  2. Ahn H., (2018b). Image-based Virtual Try-On System, Journal of Korean Computer Game Society, 31(3), 37-45.
  3. Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., and Black, M. J. (2016). Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image, European Conference on Computer Vision, pp. 561-578.
  4. Bookstein, F. L. (1989). Principal Warps: Thin-plate Splines and the Decomposition of Deformations, IEEE Transactions on PAMI, 11(6), 567-585. https://doi.org/10.1109/34.24792
  5. Cao, Z., Simon, T., Wei, S. E., and Sheikh, Y. (2017). Realtime Multi-person 2D Pose Estimation using Part Affinity Fields, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291-7299.
  6. Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. (2017). Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected Crfs, IEEE Transactions on PAMI, 40(4), 834-848.
  7. Guo, H., Miao, Z., Zhu, F., Zhang, G., and Li, S. (2014). Automatic Labanotation Generation based on Human Motion Capture Data, Chinese Conference on Pattern Recognition, pp. 426-435.
  8. Liu, F., Shen, C., and Lin, G. (2015). Deep Convolutional Neural Fields for Depth Estimation from a Single Image, Proceedings of the IEEE CVPR, pp. 5162-5170.
  9. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. (2016). SMPL: A Skinned Multi-person Linear Model, ACM Transactions on Graphics (TOG), 34(6), 248-258.
  10. Parent R. (2012). Computer Animation, Algorithms and Techniques, 3rd Edition, Morgan Kaufmann.
  11. Zanfir, M., Popa, A. I., Zanfir, A., and Sminchisescu, C. (2018). Human Appearance Transfer, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5391-5399.
  12. Weng, C. Y., Curless, B., and Kemelmacher-Shlizerman, I. (2019). Photo Wake-up: 3d Character Animation from a Single Photo, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5908-5917.