전자통신동향분석 (Electronics and Telecommunications Trends)

한국전자통신연구원 (Electronics and Telecommunications Research Institute)
권호 표지
DOI QR코드

DOI QR Code

이미지 기반 가상 착용 이미지 합성 기술 동향

A Survey of Image-based Virtual Try-on Technology

  • 박순찬 (감성디지털휴먼연구실) ;
  • 박진아 (한국과학기술원 전산학부) ;
  • 박지영 (감성디지털휴먼연구실)
  • S.C. Park ;
  • J.A. Park ;
  • J.Y. Park
  • 발행 : 2024.06.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Image synthesis has been remarkably developed in the computer vision domain and various researches have been proposed to generate realistic and high-resolution images. In particular, image-based virtual try-on is an application in fashion domain to simulate wearing clothes. Specifically, using input images of a fashion model and products, an realistic image of the model wearing the provided garments is synthesized. In this paper, we present a comprehensive review of technical trends in image-based virtual try-on technology. We first introduce relevant datasets and discuss their characteristics. Then, we categorize existing image synthesis methods into three main streams: warping-based methods, encoding-decoding-based methods, and diffusion-based methods. Finally, we explore other important research issues in the field of virtual try-on and analyze related researches aimed to tackling those challenges.

키워드

과제정보

본 연구는 과학기술정보통신부가 주관하고 한국지능정보사회진흥원이 지원하는 '인공지능 학습용 데이터 구축 사업(2차)[과제번호:2020-데이터-위64-1]'와 문화체육관광부 및 한국콘텐츠진흥원의 연구개발진흥사업[과제번호 R2020070002]으로 수행되었음.

참고문헌

  1. X. Han et al., "Viton: An image-based virtual try-on network," in Proc. CVPR, (Salt Lake City, Utah. USA), June 2018.
  2. B. Wang et al., "Toward characteristic-preserving imagebased virtual try-on network," in Proc. ECCV, (Munich, Germany), Sept. 2018.
  3. Z. Cao et al., "OpenPose: Realtime multi-person 2D pose estimation using part affinity fields," IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, 2021, pp. 172-186. https://doi.org/10.1109/TPAMI.2019.2929257
  4. https://github.com/sergeywong/cp-vton/
  5. S. Choi et al., "Viton-hd: High-resolution virtual try-on via misalignment-aware normalization," in Proc. CVPR, (Virtual), June 2021.
  6. X. Liang et al., "Look into person: Joint body parsing & pose estimation network and a new benchmark," IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 4, 2018, pp. 871-885.
  7. 박순찬 외, "복수 상품을 활용하는 고화질 패션 착용영상 생성을 위한 데이터세트 Fashion-HD 및 그 활용," 정보과학회 컴퓨팅의 실제 논문지, 제28권 제1호, 2022, pp. 68-73.
  8. D. Morelli et al., "Dress code: High-resolution multicategory virtual try-on," in Proc. ECCV, (Tel Aviv, Israel), Oct. 2022.
  9. S. Belongie, J. Malik, and J. Puzicha, "Shape matching and object recognition using shape contexts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 4, 2002, pp. 509-522. https://doi.org/10.1109/34.993558
  10. H. Yang et al., "Towards photo-realistic virtual try-on by adaptively generating-preserving image content," in Proc. CVPR, (Virtual), June 2020.
  11. S. Jandial et al., "Sievenet: A unified framework for robust image-based virtual try-on," in Proc. WACV, (Snowmass Village, CO, USA), Mar. 2020.
  12. K. Li et al., "Toward accurate and realistic outfits visualization with attention to details," in Proc. CVPR, (Virtual), June 2021.
  13. A. Chopra et al., "Zflow: Gated appearance flow-based virtual try-on with 3d priors," in Proc. ICCV, (Virtual), Oct. 2021.
  14. H. Yang, X. Yu, and Z. Liu, "Full-range virtual try-on with recurrent tri-level transform," in Proc. CVPR, (New Orleans, LA, USA), June 2022.
  15. T. Zhou et al., "View synthesis by appearance flow," in Proc. ECCV, (Amsterdam, Netherlands), Oct. 2016.
  16. X. Han et al., "Clothflow: A flow-based model for clothed person generation," in Proc. ICCV, (Seoul, Rep. of Korea), Nov. 2019.
  17. Y. Ge et al., "Parser-free virtual try-on via distilling appearance flows," in Proc. CVPR, (Virtual), June 2021.
  18. S. Bai et al., "Single stage virtual try-on via deformable attention flows," in Proc. ECCV, (Tel Aviv, Israel), Oct. 2022.
  19. S. Lee et al., "High-resolution virtual try-on with misalignment and occlusion-handled conditions," in Proc. ECCV, (Tel Aviv, Israel), Oct. 2022.
  20. S. Park and J. Park, "Single-stage virtual try-on for top and bottom clothes with wearing style control," Available at SSRN 4379142 (2023).
  21. G. Yildirim et al., "Generating high-resolution fashion model images wearing custom outfits," in Proc. ICCV, (Seoul, Rep. of Korea), Nov. 2019.
  22. A. Neuberger et al., "Image based virtual try-on network from unpaired data," in Proc. CVPR, (Virtual), June 2020.
  23. A.K. Bhunia et al., "Person image synthesis via denoising diffusion model," in Proc. CVPR, (Vancouver, Canada), June 2023.
  24. J. Sohl-Dickstein et al., "Deep unsupervised learning using nonequilibrium thermodynamics," in Proc. ICML, (Lille, France), Jul. 2015.
  25. J. Ho, A. Jain, and P. Abbeel, "Denoising diffusion probabilistic models," in Proc. NeurIPS 2020, (Virtual Only), Dec. 2020, pp. 6840-6851.
  26. P. Dhariwal and A. Nichol, "Diffusion models beat gans on image synthesis," in Proc. NeurIPS 2021, (Virtual Only), Dec. 2021, pp. 8780-8794.
  27. R. Rombach et al., "High-resolution image synthesis with latent diffusion models," in Proc. CVPR, (New Orleans, LA, USA), June 2022.
  28. D. Morelli et al., "LaDI-VTON: Latent diffusion textualinversion enhanced virtual try-on," arXiv preprint, CoRR, 2024, arXiv: 2305.13501 (2023).
  29. X. Han et al., "Controllable person image synthesis with pose-constrained latent diffusion," in Proc. ICCV, (Paris, France), Oct. 2023.
  30. J. Kim et al., "StableVITON: Learning semantic correspondence with katent diffusion model for virtual try-on," arXiv preprint, CoRR, 2023, arXiv: 2312.01725.
  31. Y. Choi et al., "Improving diffusion models for virtual try-on," arXiv preprint, CoRR, 2024, arXiv: 2403.05139.
  32. T. Park et al., "Semantic image synthesis with spatiallyadaptive normalization," in Proc. IEEE CVPR, (Long Beach, CA, USA), June 2019.
  33. L. Zhu et al., "TryOnDiffusion: A tale of two UNets," in Proc. CVPR, (Vancouver, Canada), June 2023.
  34. T.-Y. Lin et al., "Microsoft coco: Common objects in context," in Proc. ECCV, (Zurich, Switzerland), Sept. 2014
  35. C.Y. Chen et al., "Size does matter: Size-aware virtual tryon via clothing-oriented transformation try-on network," in Proc. CVPR, (Vancouver, Canada), June 2023.
  36. M. Heusel et al., "Gans trained by a two time-scale update rule converge to a local nash equilibrium," in Proc. NIPS 2017, (Long Beach, CA, USA) Dec. 2017.
  37. M. Binkowski et al., "Demystifying mmd gans," in Proc. ICLR, (Vancouver, Canada), Apr. 2018.