DOI QR코드

DOI QR Code

"Where can I buy this?" - Fashion Item Searcher using Instance Segmentation with Mask R-CNN

"이거 어디서 사?" - Mask R-CNN 기반 객체 분할을 활용한 패션 아이템 검색 시스템

  • Jung, Kyunghee (Dept. of Superintelligence, Sungkyunkwan University) ;
  • Choi, Ha nl (College of Software, Sungkyunkwan University) ;
  • Sammy, Y.X.B. (Dept. of Superintelligence, Sungkyunkwan University) ;
  • Kim, Hyunsung (Dept. of Electrical and Computer Engineering, Sungkyunkwan University) ;
  • Toan, N.D. (Dept. of Superintelligence, Sungkyunkwan University) ;
  • Choo, Hyunseung (Dept. of Electrical and Computer Engineering, Sungkyunkwan University)
  • 정경희 (성균관대학교 수퍼인텔리전스학과) ;
  • 최하늘 (성균관대학교 소프트웨어학과) ;
  • ;
  • 김현성 (성균관대학교 전기컨퓨터공학부) ;
  • ;
  • 추현승 (성균관대학교 전기컨퓨터공학부)
  • Published : 2022.11.21

Abstract

Mobile phones have become an essential item nowadays since it provides access to online platform and service fast and easy. Coming to these platforms such as Social Network Service (SNS) for shopping have been a go-to option for many people. However, searching for a specific fashion item in the picture is challenging, where users need to try multiple searches by combining appropriate search keywords. To tackle this problem, we propose a system that could provide immediate access to websites related to fashion items. In the framework, we also propose a deep learning model for an automatic analysis of image contexts using instance segmentation. We use transfer learning by utilizing Deep fashion 2 to maximize our model accuracy. After segmenting all the fashion item objects in the image, the related search information is retrieved when the object is clicked. Furthermore, we successfully deploy our system so that it could be assessable using any web browser. We prove that deep learning could be a promising tool not only for scientific purpose but also applicable to commercial shopping.

Keywords

Acknowledgement

This research was supported by National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (NRF-2020R1A2C2008447), under the ICT Creative Consilience program(IITP-2022-2020-0-01821), and Grand Information Technology Research Center support program(IITP-2022-2015-0-00742) supervised by the IITP(Institute for Information & communications Technology Planning & Evaluation)