Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning Braille Block Recognition Method for Embedded Devices

임베디드 기기를 위한 딥러닝 점자블록 인식 방법

  • 김희진 (동의대학교 컴퓨터소프트웨어공학과) ;
  • 윤재혁 (동의대학교 컴퓨터소프트웨어공학과) ;
  • 권순각 (동의대학교 컴퓨터소프트웨어공학과)
  • Received : 2023.08.08
  • Accepted : 2023.08.20
  • Published : 2023.08.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method to recognize the braille blocks for embedded devices in real time through deep learning. First, a deep learning model for braille block recognition is trained on a high-performance computer, and the learning model is applied to a lightweight tool to apply to an embedded device. To recognize the walking information of the braille block, an algorithm is used to determine the path using the distance from the braille block in the image. After detecting braille blocks, bollards, and crosswalks through the YOLOv8 model in the video captured by the embedded device, the walking information is recognized through the braille block path discrimination algorithm. We apply the model lightweight tool to YOLOv8 to detect braille blocks in real time. The precision of YOLOv8 model weights is lowered from the existing 32 bits to 8 bits, and the model is optimized by applying the TensorRT optimization engine. As the result of comparing the lightweight model through the proposed method with the existing model, the path recognition accuracy is 99.05%, which is almost the same as the existing model, but the recognition speed is reduced by 59% compared to the existing model, processing about 15 frames per second.

본 논문은 딥러닝을 통해 실시간으로 임베디드 기기에서 점자 블록을 인식하는 방법을 제안한다. 먼저 고성능 컴퓨터에서 점자 블록 인식을 위한 딥러닝 모델을 학습시키고, 임베디드 기기에 적용하기 위하여 학습 모델을 경량화 도구에 적용한다. 점자 블록의 보행 정보를 인식하기 위해 영상에서 점자블록과의 거리를 이용하여 경로를 판별하는 알고리즘을 사용한다. 임베디드 기기를 통해 촬영한 영상에서 YOLOv8 모델을 통해 점자 블록, 볼라드, 횡단보도를 검출한 후 점자블록 경로 판별 알고리즘을 거쳐 보행정보를 인식한다. 실시간으로 점자 블록을 검출하기 위해 모델 경량화 도구를 YOLOv8에 적용한다. YOLOv8 모델 가중치의 정밀도를 기존 32비트에서 8비트로 낮추고, TensorRT 최적화 엔진을 적용하여 모델의 최적화를 진행한다. 제안된 방법을 통해 경량화 된 모델을 기존 모델과 비교한 결과, 경로 인식 정확도는 99.05%로 기존 모델과 거의 차이가 없지만, 인식 속도는 기존 모델 대비 59% 단축되어 1초에 약 15개의 프레임을 처리할 수 있다.

Keywords

Acknowledgement

본 논문은 과학기술정보통신부 및 정보통신기획평가원의 지역지능화혁신인재양성(Grand ICT연구센터) 사업의 연구결과로 수행되었음(IITP-2023-2020-0-01791).

References

  1. Edgar. (2021). Roboflow: Crosswalk, https://universe.roboflow.com/edgar1019-naver-com/crosswalk-ognxu
  2. Guo, W., Xu, G. and Wang, Y. (2022). Horizontal and Vertical Features Fusion Network Based on Different Brain Regions for Emotion Recognition, Knowledge-Based Systems, 247, https://doi.org/10.1016/j.knosys.2022.108819.
  3. Jo, J. H., Song, J. S., Kwak, J. H. and Kang, M. G. (2018), Braile Block Detection System using Back-projection in Image Processing, Proceedings of KI IT Conference, 299-301.
  4. Jocher, G., Nishimura, K., Mineeva, T. and Vilarino, R. (2020). YOLOv5, https://github.com/ultralytics/yolov5.
  5. Joo, C. W., Kim, D. Y., Shen, D. F., Kim, B. S., Min, H. K., Han, Y. H. and Lee, E. H. (2003), Implementation of the Visually Impaired Travel Aids System through Unions of RFID and Braile Block, Summer Annual Conference of IEIE, 26(1), 2907-2910.
  6. Kang, J. K., Bajeneza, V., Ahn, S. Y., Sung, M. W. and Lee, Y. S. (2022). A Method to Enhance the Accuracy of Braileblock Recofnition for Walking Assistance of the Visually Impaired: Use of YOLOv5 and Analysis of Vertex Coordinates, Journal of KIISE, 49(4), 291-297, https://doi.org/10.5626/JOK.2022.49.4.291.
  7. Kang, M. W. (2022). Roboflow: Bollard, https://universe.roboflow.com/minwoo-kang/final_27.
  8. Lee, D. S., Kim, S. H. and Kwon, S. K. (2022). Guidance for Visually Impaired Person through Braile Block Detection by Deep Learning. Journal of Multimedia Information System, 9(4), 253-260. https://doi.org/10.33851/JMIS.2022.9.4.253.
  9. Lee, J. H., Kim, I. S., Kim, H. B., Lee, S. W., Jung, S. K. (2022). Case Study of YOLOv5 Model Optimization for Edge AI, Proceedings of Symposium of the Korean Institute of Communications and Information Sciences, 78(1), 897-898.
  10. Lee, Y., Park, K. and Kwon, S. K. (2018), Implementation of Signboard Voice Guidance Service for Visually Impaired Person Using Virtual Beacon, Journal of the Korea Industrial Information Systems Research, 23(6), 1-8.
  11. Murad, M., Rehman, A., Shah, A. A., Ullah, S., Fahad, M. and Yahya, K. M. (2011). RFAIDE - An RFID Based Navigation and Object Recognition Assistant for Visually Impaired People, 7th International Conference on Emerging Technologies, Islamabad, Pakistan, pp.1-4, 10.1109/ICET. 2011.6048486.
  12. National Information Society Agency. (2021). AIHub: Sidewalk Walking Video, https://aihub.or.kr/aihubdata/data/view.do?currMenu=&topMenu=&aihubDataSe=realm&dataSetSn=189.
  13. Oh, S. J. (2017). A Study on the Braile-block Detection using Artificial Neural Networks, Graduate School of Engineering Hanyang University, Seoul, Korea.
  14. Park, I. J. and Park, D. J. (2010). A Study on Crosswalk Guidance System for the Blind using RFID, Journal of The Institute of Electronics Engineers of Korea Computer and Information, 47(6), 124-130.
  15. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016). You Only Look Once: Unified, Real-time Object Detection, Conference Computer Vision and Pattern Recognition, Dec. 779-788.
  16. Terven, J. and Cordova-Esparza, D. (2023), A Comprehensive Review of YOLO:From YOLOv1 to YOLOv8 and Beyond, arXiv, https://doi.org/10.48550/arXiv.2304.00501.
  17. Valladares, S., Toscano, M., Tufino, R., Morillo, P. and Vallefo, D. (2021). Performance Evaluation of the Nvidia Jetson Nano through a Real-Time Machine Learning Application, Proceedings of the 4th International Conference on IHSI 2021, 343-349, 10.1007/978-3-030-68017-6_51.
  18. Yang, H. J., Jang, S. W., Park, J. S., Kim, J. G. and Kwon, J. W. (2022). A Study on the Development of Real-Time Embedded Assistance System Based on Deep Learning for the Blind, Journal of Rehabilitation Welfare Engineering & Assistive Technology, 16(1), 27-36, https://doi.org/10.21288/resko.2022.16.1.27.
  19. Zhao, X., Wang, Y., Cai, X., Liu, C., and Zhang, L. (2020). Linear Symmetric Quantization of Neural Networks for Low-precision Integer Hardware, International Conference on Learning Representations.