DOI QR코드

DOI QR Code

Cloth Product Recognition based on Siamese Network with Body Region Extraction method

  • Received : 2022.05.03
  • Accepted : 2022.05.10
  • Published : 2022.06.30

Abstract

Nowadays, people consume a lot of content such as web dramas or K-pop videos through mobile devices such as smartphones, and the market for indirect advertisements through these web dramas or K-pop videos is also increasing every year. In order to lead to the immediate purchase of indirect products in web dramas, a system that allows consumers to purchase immediately at the time the products appear in the drama is needed. In this paper, we propose a system to allow viewers to purchase products worn by celebrities immediately when viewers see and click on them. When a user clicks on a video, it recognizes the product worn by the celebrity, and displays information on the screen on the most similar product corresponding to the recognized product, allowing them to go to the seller's site where they can purchase it. In order for such a system to operate stably, a pose estimation and siamese network-based system is proposed. The proposed system will primarily be released as a streaming service in the form of an app or web page that connects the products in web dramas or other K-pop video contents screened on the mobile with e-commerce. Furthermore, in the future, the technology is expected to be used globally in various industries such as smart mobility and display kiosks.

Keywords

Acknowledgement

This work was supported by the Technology development Program(S2840023) funded by the Ministry of SMEs and Startups(MSS, Korea).

References

  1. L. Bertinetto, J. Valmadre, J.F. Henriques, A. Vedaldi, and P.H.S. Torr, "Fully-Convolutional Siamese Networks for Object Tracking," in European Conference on Computer Vision (ECCV) 2016, Oct. 8-16, 2016. DOI: https://doi.org/10.1007/978-3-319-48881-3_56
  2. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, "DeepFace: Closing the Gap to Human-Level Performance in Face Verification," in 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1701-1708, Jun. 23-28, 2014. DOI: https://doi.org/10.1109/CVPR.2014.220
  3. Y. Cheng, Y. Tian, and M. He, "Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods," Computer Vision and Image Understanding (CVIU), Vol. 192, 102897, March 2020. DOI: https://doi.org/10.1016/j.cviu.2019.102897
  4. Y. Wu, A. Kirillov, F. Massa, W.Y. Lo, and R.Girshick, "Detectron2," https://github.com/facebookresearch/detectron2
  5. S. Ren, K. He, R. Girshick, J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, pp.1137-1149, June, 2017. DOI: https://doi.org/10.1109/TPAMI.2016.2577031