The Journal of Korea Institute of Information, Electronics, and Communication Technology (한국정보전자통신기술학회논문지)

Korea Information Electronic Communication Technology (한국정보전자통신기술학회)
권호 표지
DOI QR코드

DOI QR Code

Automation of Online to Offline Stores: Extremely Small Depth-Yolov8 and Feature-Based Product Recognition

Online to Offline 상점의 자동화 : 초소형 깊이의 Yolov8과 특징점 기반의 상품 인식

  • Jongwook Si (Dept. Computer.AI Convergence Engineering, Kumoh National Institute of Technology) ;
  • Daemin Kim (Dept. Computer Engineering, Kumoh National Institute of Technology) ;
  • Sungyoung Kim (Dept. Computer Engineering, Kumoh National Institute of Technology)
  • 시종욱 ;
  • 김대민 ;
  • 김성영
  • Received : 2024.05.10
  • Accepted : 2024.05.19
  • Published : 2024.06.29
Download PDF
⟨ Previous Next ⟩

Abstract

The rapid advancement of digital technology and the COVID-19 pandemic have significantly accelerated the growth of online commerce, highlighting the need for support mechanisms that enable small business owners to effectively respond to these market changes. In response, this paper presents a foundational technology leveraging the Online to Offline (O2O) strategy to automatically capture products displayed on retail shelves and utilize these images to create virtual stores. The essence of this research lies in precisely identifying and recognizing the location and names of displayed products, for which a single-class-targeted, lightweight model based on YOLOv8, named ESD-YOLOv8, is proposed. The detected products are identified by their names through feature-point-based technology, equipped with the capability to swiftly update the system by simply adding photos of new products. Through experiments, product name recognition demonstrated an accuracy of 74.0%, and position detection achieved a performance with an F2-Score of 92.8% using only 0.3M parameters. These results confirm that the proposed method possesses high performance and optimized efficiency.

디지털 기술의 급속한 발전과 코로나19 팬데믹으로 인해 온라인 상거래가 크게 성장하면서, 소상공인들이 이러한 시장 변화에 적극적으로 대응할 수 있는 지원 방안의 필요성이 대두되었다. 이에 본 논문은 O2O(Online to Offline) 전략을 활용해 실제 매장 진열대에 전시된 상품들을 자동으로 촬영하고 이를 이용해 가상 상점을 만들 수 있는 기초적인 기술을 제시한다. 본 연구의 핵심은 진열된 상품의 위치와 이름을 정확히 파악하여 인식하는 것이며, 이를 위해 단일 클래스를 대상으로 하며 YOLOv8에 기반한 경량화 모델인 ESD-YOLOv8을 제안한다. 검출된 상품은 특징점 기반의 기술을 통해 상품명이 식별되며, 이는 새 상품을 사진 형태로 추가함으로써 신속하게 갱신할 수 있는 능력을 갖추고 있다. 실험을 통해 상품명 인식은 74.0%의 정확도, 위치 검출은 0.3M개의 파라미터만으로 F2-Score 기준 92.8%의 성능을 보였다. 이를 통해 제안된 방법이 높은 성능과 최적화된 효율성을 갖추고 있음을 확인하였다.

Keywords

Acknowledgement

This work was supported by the Technology development Program(S3344882) funded by the Ministry of SMEs and Startups(MSS, Korea)

References

  1. https://www.sisaweek.com/news/articleView.html?idxno=155389 
  2. https://www.segye.com/newsView/20221212516736 
  3. D. Kim, J. Si, S. Lee, and S. Kim, "Calculation of Product Location Based on Object Detection and Product name recognition through Image Similarity Measurement", Proceedings of KIIT Conference, pp.494-495, 2023. 
  4. D. Kim, J. Si, and S. Kim, "Feature Point Matching for Product Name Recognition in O2O Stores", Proceedings of KSCI Conference, pp.79-80, 2024. 
  5. YOLOv8, https://github.com/ultralytics/ultralytics 
  6. J. Redmon, S. Divvala, and R. Girshick, "You Only Look Once: Unified, Real-Time Object Detection", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016. 
  7. W. Liu, A. C. Berg, et al., "SSD: Single Shot MultiBox Detector", European Conference on Computer Vision, pp. 21-37, 2016. 
  8. T. Y. Lin, P. Dollar et al., "Focal loss for dense object detection", In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017. 
  9. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Region-Based Convolutional Networks for Accurate Object Detection and Segmentation", IEEE transactions on pattern analysis and machine intelligence, Vol 38, No. 1, pp. 142-158, 2015. 
  10. R. GIRSHICK, "Fast r-cnn", Proceedings of the IEEE international conference on computer vision, pp.1440-1448, 2015. 
  11. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", Vol. 39, No. 6, pp.1137-1149, 2015. 
  12. J. Si, G. Kim, J. Kim, and S. Kim, "Enhanced Location-based Facility Management in Mobile Environments using Object Recognition and Augmented Reality", The Journal of Korean Institute of Information Technology, Vol. 21, No. 11, pp. 183-192, 2023. 
  13. J. Si, M. Kim, and S. Kim, "Converting Close-Looped Electronic Circuit Image with Single I/O Symbol into Netlist", The Journal of Korean Institute of Information Technology, Vol. 19, No. 8, pp. 1-10, 2021. 
  14. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints", International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004. 
  15. H. Bay, T. Tuytelaars, and L. Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding, Vol. 110, No. 3, pp. 346-359, 2008. 
  16. E. Rublee, V. Rabaud, K. Konolige and G. Bradski, "ORB: An efficient alternative to SIFT or SURF", International Conference on Computer Vision, pp. 2564-2571, 2011. 
  17. M. Calonder, V. Lepetit, C. Strecha, and P. Fua. "Brief: Binary robust independent elementary features", European Conference on Computer Vision, pp. 778-792, 2010.