Browse > Article
http://dx.doi.org/10.5392/JKCA.2022.22.10.001

Object Detection Algorithm for Explaining Products to the Visually Impaired  

Park, Dong-Yeon (숙명여자대학교 IT공학과)
Lim, Soon-Bum (숙명여자대학교 IT공학과)
Publication Information
The Journal of the Korea Contents Association / v.22, no.10, 2022 , pp. 1-10 More about this Journal
Abstract
Visually impaired people have very difficulty using retail stores due to the absence of braille information on products and any other support system. In this paper, we propose a basic algorithm for a system that recognizes products in retail stores and explains them as a voice. First, the deep learning model detects hand objects and product objects in the input image. Then, it finds a product object that most overlapping hand object by comparing the coordinate information of each detected object. We determine that this is a product selected by the user, and the system read the nutritional information of the product as Text-To-Speech. As a result of the evaluation, we confirmed a high performance of the learning model. The proposed algorithm can be actively used to build a system that supports the use of retail stores for the visually impaired.
Keywords
Object Detection; Hand Detection; Visually Impaired; Retail Store; Text-To-Speech;
Citations & Related Records
연도 인용수 순위
  • Reference
1 https://www.jnilbo.com/view/media/view?code=2022041711264849598.html
2 BeYerage, 시각장애인을 위한 편의점 음료 안내 서비스, 공개SW 개발자대회 결과보고서, 2021.
3 T. Simon, H. Joo, I. Matthews, and Y. Sheikh, "Hand Keypoint Detection in Single Images using Multiview Bootstrapping," Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.1145-1153, 2017.
4 https://aihub.or.kr/aidata/34145
5 https://roboflow.com/
6 C. Y. Wang, H. Y. Mark Liao, P. Y. Chen, and J. W. Hsieh, "Enriching variety of layer-wise learning information by gradient combination," Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
7 X. Zhu, S. Lyu, X. Wang, and Q. Zhao, "TPH-YOLOv5: Improved YOLOv5 based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios," Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.2778-2788, 2021.
8 D. S. Berry, "Developments in the AST Library," Astronomical Data Analysis Software and Systems XVII, Vol.394, pp.635-637, 2008.
9 S. Virtue and A. Vidal-Puig, "GTTs and ITTs in mice: simple tests, complex answers," Nature Metabolism, Vol.3, No.7, pp.883-886, 2021.   DOI
10 김재훈, 김수종, 이다예, 전민건, 김재현, 전수빈, 서동만, "시각장애인을 위한 인공지능 기반 전방 물체 탐지," 한국정보과학회 학술발표논문집, pp.1978-1980, 2021.
11 S. Bambach, S. Lee, D. J. Crandall, and C. Yu, "Lending a hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions," Proceeding of IEEE International Conference of Computer Vission, pp.1949-1957, 2015.
12 최경민, 김민영, 황수영, 오유군, 임덕신, "시각장애인의 안전한 요리를 위한 주방 비전 센싱 시스템," 대한인간공학회 학술대회논문집, pp.355-356, 2021.
13 오세랑, 배영철, "YOLO v3 기반의 시각장애인을 위한 유도 블록 인식 알고리즘," 한국지능시스템학회 논문지, 제31권 제1호, pp.60-67, 2021.
14 2021년 12월, 2021년 주요 유통사별 매출 추이, 한국산업통상자원부, 2022.
15 https://news.mt.co.kr/photonews/graphicNewsList.html?id=2021010709573328767.html
16 배민경, 서현아, 이영우, "OCR 기반 시각장애인을 위한 편의점 음료 구입 보조 시스템," 한국콘텐츠학회종합학술대회 논문집, pp.11-12, 2021.
17 https://github.com/DyeonPark/Product-Detect ion-for-Visually-Impaired
18 https://github.com/cansik/yolo-hand-detection