Browse > Article
http://dx.doi.org/10.9723/jksiis.2020.25.5.001

3D Reconstruction of a Single Clothing Image and Its Application to Image-based Virtual Try-On  

Ahn, Heejune (서울과학기술대학교 전기정보공학과)
Minar, Matiur Rahman (서울과학기술대학교 전기정보공학과)
Publication Information
Journal of Korea Society of Industrial Information Systems / v.25, no.5, 2020 , pp. 1-11 More about this Journal
Abstract
Image-based virtual try-on (VTON) is becoming popular for online apparel shopping, mainly because of not requiring 3D information for try-on clothes and target humans. However, existing 2D algorithms, even when utilizing advanced non-rigid deformation algorithms, cannot handle large spatial transformations for complex target human poses. In this study, we propose a 3D clothing reconstruction method using a 3D human body model. The resulting 3D models of try-on clothes can be more easily deformed when applied to rest posed standard human models. Then, the poses and shapes of 3D clothing models can be transferred to the target human models estimated from 2D images. Finally, the deformed clothing models can be rendered and blended with target human representations. Experimental results with the VITON dataset used in the previous works show that the shapes of reconstructed clothing are significantly more natural, compared to the 2D image-based deformation results when human poses and shapes are estimated accurately.
Keywords
3D Model reconstruction; Skinned Multi-Person Linear (SMPL) model; 3D Reconstruction; Garment model; Virtual Try-On (VTON);
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., and Black, M. J. (2016). Keep It SMPL: Automatic Estimation of 3d Human Pose and Shape from a Single Omage, European Conference on Computer Vision, pp. 561-578
2 Bookstein, F. L. (1989). Principal Warps: Thin-plate Splines and the Decomposition of Deformations, IEEE Trans. on PAMI 11(6), 567-585, DOI: 10.1109/34.24792   DOI
3 Cao, Z., Martinez, G. H., Simon, T., Wei, S. E., and Sheikh, Y. (2018). Realtime Multi-person 2d Pose Estimation using Part Affinity Fields, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291-7299.
4 Han, X., Wu, Z., Wu, Z., Yu, R., and Davis, L. S. (2018). VITON: An Image-based Virtual Try-on Network, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7543-7552.
5 Liang, X., Gong, K., Shen, X., and Lin, L. (2018). Look into Person: Joint Body Parsing and Pose Estimation Network and a New Benchmark, IEEE Trans on PAMI. 41, 871-885. https://doi.org/10.1109/TPAMI.2018.2820063   DOI
6 Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. (2015). Smpl: A Skinned Multi-person Linear Model, ACM Trans on Graphics, 34, 248-248. https://doi.org/10.1145/2816795.2818013
7 Mir, A., Alldieck, T., and Pons-Moll, G. (2020). Learning to Transfer Texture from Clothing Images to 3d Humans, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7023-7034.
8 Patel, C., Liao, Z., and Pons-Moll, G. (2020). Tailornet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7365-7375.
9 Pishchulin, L., Insafutdinov, E., Tang, S., Andres, B., Andriluka, M., Gehler, P. V., and Schiele, B. (2016). Deepcut Joint Subset Partition and Labeling for Multi Person pose Estimation, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4929-4937
10 Ahn H., (2018). Image-based Virtual Try-On System: Full Automatic System Design and its Performance, J ournal of Korean Computer Game Society, 31(3), 37-45.
11 Belongie, S., Malik, J., and Puzicha, J. (2002). Shape Matching and Object Recognition using Shape Contexts, IEEE Trans. on PAMI 24(4), 509-522, https://doi.org/10.1109/34.993558   DOI
12 Bhatnagar, B. L., Tiwari, G., Theobalt, C., and Pons-Moll, G. (2019). Multi-garment Net: Learning to Dress 3d People from Images, Proc. of the IEEE International Conference on Computer Vision. pp. 5420-5430.
13 Weng, C. Y., Curless, B., and Kemelmacher-Shlizerman, I. (2019). Photo Wake-up: 3d Character Animation from a Single Photo, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5908-5917
14 Pons-Moll, G., Pujades, S., Hu, S., and Black, M. J. (2017). Clothcap: Seamless 4d Clothing Capture and Retargeting, ACM Trans. on Graphics. 36(4), 1-15 https://doi.org/10.1145/3072959.3073711
15 Sun, F.D., Guo, J., Su, Z., and Gao, C. Y. (2019). Image-based Virtual Try-on Network with Structural Coherence, Proc. of the IEEE International Conference on Image Processing, pp. 519-523
16 Wang, B., Zhang, H., Liang, X., Chen, Y., Lin, L., and Yang, M. (2018). Toward Characteristic Preserving Image-based Virtual Try-on Network, European Conference on Computer Vision, pp. 589-604.
17 Yu, R., Wang, X., and Xie, X. (2019). VTNFP: An Image-based Virtual Try-on Network with Body and Clothing Feature Preservation, Proc. of the IEEE International Conference on Computer Vision, pp. 10511-10520.
18 Zanfir, M., Popa, A. I., Zanfir, A., and Sminchisescu, C. (2018). Human Appearance Transfer, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5391-599.