Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Online Virtual Try On using Mannequin Cloth Pictures

마네킨 의상사진 기반 온라인 가상의상착용

  • 안희준 (서울과학기술대학교 전기정보공학과)
  • Received : 2018.11.26
  • Accepted : 2018.12.16
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we developed a virtual cloth try-on (VTON) technology that segement the cloth image worn on the mannequin and applies it to the user 's photograph. The two-dimensional image-based virtual wear study which does not require three-dimensional information of cloth and model is of practical value, but the research result shows that there are limitations of of the current technology for the problem of occlusion or distortion. In this study, we proposed an algorithm to apply the results obtained from the DNN- based segmentation and posture estimation to the user 's photograph, assuming that the mannequin cloth reduces the difficulties in this part. In order to improve the performance compared with the existing one, we used the validity check of the pre-attitude information, the improvement of the deformation using the outline, and the improvement of the divided area. As a result, a significantly improved result image of more than 50% was obtained.

본 논문에서 마네킨에 착용된 의상 이미지를 분할하고 사용자의 사진에 입히는 가상의상착용 (VTON) 기술을 개발하였다. 의상과 모델의 3차원 정보가 필요하지 않는 2차원 이미지 기반 가상착용연구는 실용적인 가치가 크지만, 연구결과 현재 기술로는 의상 분할 시 가림이나 왜곡에 의한 문제 등 제약사항이 존재한다. 본 연구는 마네킨 의상을 사용함으로써 이러한 어려움을 줄였다는 가정 하에서, 딥러닝 기반 영역분할과 자세추정을 통하여 얻은 결과를 사용자 사진에 입히는 알고리즘을 제안하였다. 기존의 연구 대비 성능 개선을 위하여 사전 자세정보의 신뢰성 검사, 외곽선을 이용한 변형개선, 분할 영역개선 등을 사용하였다. 결과로 시각적으로 만족할 만한 의상착용의 경우가 전체의 50%이상으로 상당히 개선된 결과를 얻었다.

Keywords

SOJBB3_2018_v23n6_29_f0001.png 이미지

Fig. 1 Model Cloth Based VTON System Results [5] (Top: Successful, Middle and Bottom: Unsuccessful)

SOJBB3_2018_v23n6_29_f0002.png 이미지

Fig. 2 VTON System Application Scenario Example

SOJBB3_2018_v23n6_29_f0003.png 이미지

Fig. 3 Parsing Results (Success and Fail) [5]

SOJBB3_2018_v23n6_29_f0004.png 이미지

Fig. 4 Pose Estimation Results

SOJBB3_2018_v23n6_29_f0005.png 이미지

Fig. 5 Detected Wrong Pose Estimated Cased

SOJBB3_2018_v23n6_29_f0006.png 이미지

Fig. 6 Estimation Method for the Hip Boundary Location

SOJBB3_2018_v23n6_29_f0007.png 이미지

Fig. 7 Boundary Key Points Extracted (left:short clothes, middle: long upper cloth and pants, right: skirt)

SOJBB3_2018_v23n6_29_f0008.png 이미지

Fig. 8 The Input and Final VTON Image (Enlarged) (Left: Upper and Skirt, Right Upper and Pants)

SOJBB3_2018_v23n6_29_f0009.png 이미지

Fig. 9 Boundary Enhancement and Blending Effects (Left: Before, Right: After)

SOJBB3_2018_v23n6_29_f0010.png 이미지

Fig. 10 VTON Results with Uncovered Area from Input Clothes

Table 1 Joints and Labels

SOJBB3_2018_v23n6_29_t0001.png 이미지

References

  1. Yamaguchi, K., Kiapour, M. H., Ortiz, L. E., and Berg, T. L., "Parsing Clothing in Fashion Photographs," in Proc. of Computer Vision and Pattern Recognition (CVPR), 2012.
  2. Yamaguchi, K., Kiapour, M. H., and Berg, T. L., "Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing Items," in Proc. of International Conference Computer Vision (ICCV), 2013.
  3. Kim, J., and Forsythe, S., "Adoption of Virtual Try-on Technology for Online Apparel Shopping," Journal of Interactive Marketing, Vol. 22, No. 2, pp. 45-59, 2008. https://doi.org/10.1002/dir.20113
  4. Liang, X., Liu, S., Shen, X., Yang, J., Liu, L., Dong, J., and Yan, S., "Deep Human Parsing with Active Template Regression," IEEE Tran. on Pattern Analysis and Machine Intelligence, Vol. 37, No. 12, pp. 2402-2414, 2015. https://doi.org/10.1109/TPAMI.2015.2408360
  5. Ahn, H., "Image-Based Virtual Try-On System," J. of Korean Computer Game Society, Vol. 31, No 3, pp. 37-45, 2018.
  6. Johnson S., and Everingham, M., "Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation," In Proc. of the 21st British Machine Vision Conference (BMVC2010, 2010.
  7. Andriluka, M., Pishchulin, L., Gehler, P., and Bernt, S., "2D Human Pose Estimation: New Benchmark and State of the Art Analysis," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
  8. Long, J., Shelhamer, E., and Darrell, T., "Fully Convolutional Networks for Semantic Segmentation," In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, 2015.
  9. Vijay, B., Kendall, A., and Cipolla, R., "Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495, 2017. https://doi.org/10.1109/TPAMI.2016.2644615
  10. Krahenbuhl, P. and Koltun, V., "Efficient Inference in Fully Connected Crfs with Gaussian Edge Potentials," In Advances in Neural Information Processing Systems, 2011.
  11. Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., and Schiele, B., "DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model," In European Conference on Computer Vision (ECCV), 2016.
  12. Insafutdinov, E., Andriluka, M., Pishchulin, L., Tang, S., Levinkov, E., Andres, B., and Schiele, B., "ArtTrack: Articulated Multi-person Tracking in the Wild," In Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  13. Bookstein, F. L., "Principal Warps: Thin-Plate Splines and the Decomposition of Deformations," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 11, No. 6, pp. 567-585, 1989. https://doi.org/10.1109/34.24792
  14. Igarashi, T., Moscovich, T., and Hughes, J. F., "As-Rigid-as-possible Shape Manipulation," ACM Transactions on Graphics, Vol. 24, No. 3, pp. 1134-1141, 2005. https://doi.org/10.1145/1073204.1073323
  15. Kim, N., Yoon, J. C., and Lee, I. K., "Image-Based Dress up System," In ACM SIGGRAPH'09, pp. 35, 2009.
  16. Rother, C., Kolmogorov, V., and Blake, A., "Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts," In ACM Transactions on Graphics (TOG), Vol. 23, No. 3, pp. 309-314, 2004. https://doi.org/10.1145/1015706.1015720
  17. Baudisch, P., and Gutwin, C., "Multiblending: Displaying Overlapping Windows Simultaneously without the Drawbacks of Alpha Blending," In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ACM, pp. 367-374, 2004.
  18. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., and Yang, M., "Toward Characteristic-Preserving Image-Based Virtual Try-On Network," In Proc. of the European Conference on Computer Vision (ECCV), pp. 589-604, 2018.
  19. Lee, J. H., Kim, B. M., and Shin, Y. S., "Effects of Preprocessing and Feature Extraction on CNN-Based Fire Detection Performance," J. Korea Industrial Information System Research, Vol. 23, No. 4, pp. 41-53, 2018. https://doi.org/10.9723/JKSIIS.2018.23.4.041
  20. Kim, D. H. Choi, S. W., and Kwak, S. Y., "Deep Learning Based Fake Face Detection," J. Korea Industrial Information System Research, Vol. 23, No. 5, pp. 9-17, 2018. https://doi.org/10.9723/jksiis.2018.23.5.009
  21. Ahn, H., "A Study on Compression of Connections in Deep Artificial Neural Networks," J. Korea Industrial Information System Research, Vol. 22, No. 5. pp. 17-24, 2017. https://doi.org/10.9723/JKSIIS.2017.22.5.017

Cited by

  1. 이미지를 사용한 가상의상착용 알고리즘들의 성능 분석 vol.24, pp.6, 2018, https://doi.org/10.9723/jksiis.2019.24.6.025
  2. 이미지를 사용한 가상의상착용을 위한 개선된 알고리즘 vol.25, pp.2, 2018, https://doi.org/10.9723/jksiis.2020.25.2.011