Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Driver Drowsiness Detection System using Image Recognition and Bio-signals

영상 인식 및 생체 신호를 이용한 운전자 졸음 감지 시스템

  • Lee, Min-Hye (Center for General Education, Wonkwang University) ;
  • Shin, Seong-Yoon (School of Computer Information & Communication Engineering, Kunsan National University)
  • Received : 2022.04.26
  • Accepted : 2022.05.14
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Drowsy driving, one of the biggest causes of traffic accidents every year, is accompanied by various factors. As a general method to check whether or not there is drowsiness, a method of identifying a driver's expression and driving pattern, and a method of analyzing bio-signals are being studied. This paper proposes a driver fatigue detection system using deep learning technology and bio-signal measurement technology. As the first step in the proposed method, deep learning is used to detect the driver's eye shape, yawning presence, and body movement to detect drowsiness. In the second stage, it was designed to increase the accuracy of the system by identifying the driver's fatigue state using the pulse wave signal and body temperature. As a result of the experiment, it was possible to reliably determine the driver's drowsiness and fatigue in real-time images.

매년 교통사고의 가장 큰 원인으로 손꼽히는 졸음운전은 운전자의 수면 부족, 산소 부족, 긴장감의 저하, 신체의 피로 등과 같은 다양한 요인을 동반한다. 졸음 유무를 확인하는 일반적인 방법으로 운전자의 표정과 주행패턴을 파악하는 방법, 심전도, 산소포화도, 뇌파와 같은 생체신호를 분석하는 방법들이 연구되고 있다. 본 논문은 영상을 검출하는 딥러닝 모델과 생체 신호 측정 기술을 이용한 운전자 피로 감지 시스템을 제안한다. 제안 방법은 일차적으로 딥러닝을 이용하여 운전자의 눈 모양과 하품 유무, 졸음으로 예상되는 신체 동작을 파악하여 졸음 상태를 감지한다. 이차적으로 맥파 신호와 체온을 이용하여 운전자의 피로 상태를 파악하여 시스템의 정확도를 높이도록 설계하였다. 실험 결과, 실시간 영상에서 운전자의 졸음 유무 판별이 안정적으로 가능하였으며 각성상태와 졸음 상태에서의 분당 심박수와 체온을 비교하여 본 연구의 타당성을 확인할 수 있었다.

Keywords

Acknowledgement

This paper was supported by Wonkwang University in 2021.

References

  1. Y. H. Kim, C. H. Lee, H. S. Cho, J. S. Park, J. Y. Park, G. H. Song, J. H. An, S. J. Lee, J. Y. Lee, J. S. Lee, and S. J. Hong, "Convergent Ideation for Future Transport Systems," Korea transport institute(KOTI), Sejong, KR, Research Report, 2011.
  2. B. T. Ahn, "Study for Drowsy Driving Detection & Prevention System," Journal of Convergence for Information Technology, vol. 8, no. 3, pp. 193-198, Mar. 2018. https://doi.org/10.22156/CS4SMB.2018.8.3.193
  3. Public data portal. National Police Agency_Drowsy Driving Traffic Accidents [Internet]. Available: https://www.data.go.kr/data/15047952/fileData.do.
  4. S. G. Lee, Y. S. Kwon, J. S. Park, S. J. Yun, and W. T. Kim, "A Sleep-driving Accident Prevention System based on EEG analysis using Deep-learning Algorithm," Journal of The Institute of Electronics and Information Engineers, vol. 55, no. 3, pp. 67-73, Mar. 2018 https://doi.org/10.5573/ieie.2018.55.3.67
  5. M. Y. Oh, Y. S. Jeong, and K. H. Park, "Driver Drowsiness Detection Algorithm based on Facial Feature," Journal of Korea Multimedia Society, vol. 19, no. 11, pp. 1852-1861, Nov. 2016. https://doi.org/10.9717/KMMS.2016.19.11.1852
  6. H. A. Lee and S. Y. Shin, "Implementation of Drowsy Prevention System Using Arduino and YOLO," Journal of the Korea Institute of Information and Communication Engineering, vol. 25, no. 7, pp. 917-922, Jul. 2021. https://doi.org/10.6109/JKIICE.2021.25.7.917
  7. J. W. Son and M. O. Park, "Driving behavior Analysis to Verify the Criteria of a Driver Monitoring System in a Conditional Autonomous Vehicle-Part II-," Journal of Auto-Vehicle Safety Association, vol. 13, no. 1, pp. 38-44, Mar. 2022.
  8. H. T. Choi, M. K. Back, J. S. Kang, and K. C. Lee, "Driver Drowsiness Detection Based on Visual-Feature Using Multi-Modal Learning," The Journal of Korean Institute of Communications and Information Sciences, vol. 43, no. 7, pp. 1124-1132, Jul. 2018. https://doi.org/10.7840/kics.2018.43.7.1124
  9. S. M. Jeong, G. H. Kim, H. J. Mun, and C. G. Kim, "Design and Implementation of a System to Detect Zigzag Driving using Sensor," Journal of digital convergence, vol. 14, no. 11, pp. 305-311, Nov. 2016. https://doi.org/10.14400/JDC.2016.14.11.305
  10. B. J. Moon, K. B. Yeon, S. G. Lee, S. P. Hong, S. Y. Nam, and D. H. Kim, "Drowsy Driving Detection Algorithm Using a Steering Angle Sensor And State of the Vehicle," Journal of the institute of electronics engineers of Korea IE, vol. 49, no. 2, pp. 30-39, Jun. 2012.
  11. C. M. Park, "A Study on the Drowsy Driving Prevention System using the Pulse Sensor," in Proceeding of the 38th Korea Institute of Information and Communication Engineering, Busan, Korea, pp. 577-578, 2016.
  12. K. Fujiwara, E. Abe, K. Kamata, C. Nankayama, Y. Suzuki, T. Yamakawa, T. Hiraoka, M. Kano, Y. Sumi, F. Masuda, M. Matsuo, and H. Kadotani, "Heart rate variability-based driver drowsiness detection and its validation with EEG," IEEE Transactions on Biomedical Engineering, vol. 66, no. 6, pp. 1769-1778, Jun. 2019. https://doi.org/10.1109/tbme.2018.2879346
  13. J. Y. Lee, J. H. Jeong, D. Y. Kim, J. H. Gwon, and T. J. Yun, "Drowsiness warning system using eye-blink and heart rate," in Proceeding of the 64th Korea Society of Computer and Information, Jeju, Korea, pp. 519-520, 2021.
  14. H. S. Park, "Appratus and method for sensing driver sleepiness/drinking," Hyundai Mobis, Seoul, Korea, Patent 10-2011-0093033, DOI: 10.8080/1020110093033.