Browse > Article
http://dx.doi.org/10.9708/jksci.2021.26.11.173

Study on driver's distraction research trend and deep learning based behavior recognition model  

Han, Sangkon (Dept. of Computer Science Engineering, Pusan National University)
Choi, Jung-In (Dept. of Applied Artificial Intelligence, Ajou University)
Publication Information
Journal of the Korea Society of Computer and Information / v.26, no.11, 2021 , pp. 173-182 More about this Journal
Abstract
In this paper, we analyzed driver's and passenger's motions that cause driver's distraction, and recognized 10 driver's behaviors related to mobile phones. First, distraction-inducing behaviors were classified into environments and factors, and related recent papers were analyzed. Based on the analyzed papers, 10 driver's behaviors related to cell phones, which are the main causes of distraction, were recognized. The experiment was conducted based on about 100,000 image data. Features were extracted through SURF and tested with three models (CNN, ResNet-101, and improved ResNet-101). The improved ResNet-101 model reduced training and validation errors by 8.2 times and 44.6 times compared to CNN, and the average precision and f1-score were maintained at a high level of 0.98. In addition, using CAM (class activation maps), it was reviewed whether the deep learning model used the cell phone object and location as the decisive cause when judging the driver's distraction behavior.
Keywords
Driver's Behavior; Driver's Distraction; Behavior Recognition; ResNet-101; CAM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Q. Hua, L. Jin, Y. Jiang, M. Gao and B. Guo, "Cognitive Distraction State Recognition of Drivers at a Nonsignalized Intersection in a Mixed Traffic Environment," Advances in Civil Engineering, vol. 2021, pp. 1-16, Mar 4, 2021. DOI: 10.1155/2021/6676807.   DOI
2 J. H. L. Hansen, C. Busso, Yang Zheng and A. Sathyanarayana, "Driver Modeling for Detection and Assessment of Driver Distraction: Examples from the UTDrive Test Bed," EEE Signal Processing Magazine, vol. 34, pp. 130-142, Jul. 2017.
3 N. Li, and C. Busso, "Predicting perceived visual and cognitive distractions of drivers with multimodal features," IEEE Transactions on Intelligent Transportation Systems, vol. 16, (1), pp. 51-65, Feb, 2015. DOI: 10.1109/TITS.2 014.2324414.   DOI
4 C. Huisingh, R. Griffin, and G. McGwin, "The prevalence of distraction among passenger vehicle drivers: a roadside observational approach," Traffic injury prevention, vol. 16, (2), pp. 140-146, Feb 17, 2015. DOI: 10.1080/15389588.20 14.916797.   DOI
5 W. Yan, S. Peng, C. Li, and L. Yang, "Impact to Longitude Velocity Control of Autonomous Vehicle from Human Driver's Distraction Behavior," Proceeding of IEEE 86th Vehicular Technology Conference (VTC-Fall), in Sep 2017, pp. 1-5.
6 G. Gunduz, C. Yaman, A. U. Peker and T. Acarman, "Prediction of Risk Generated by Different Driving Patterns and Their Conflict Redistribution," Tiv, vol. 3, (1), pp. 71-80, Mar, 2018. DOI: 10.1109/TIV.2017.2788203.   DOI
7 M. N. Azadani and A. Boukerche, "Driving Behavior Analysis Guidelines for Intelligent Transportation Systems," IEEE Transactions on Intelligent Transportation Systems, pp. 1-19, 2021. DOI: 10.1109/TITS.2021.3076140.   DOI
8 S. Kim, "A Study of Driver's Visual and Auditory Task Distraction using Virtual Reality Driving Simulator," Master Thesis, Soongsil University, Korea, 2018.
9 X. Zeng and J. Wang, "A stochastic driver pedal behavior model incorporating road information," IEEE Transactions on Human-Machine Systems, vol. 47, (5), pp. 614-624, 2017.   DOI
10 M. Dua, Shakshi, R. Singla, S. Raj and A. Jangra, "Deep CNN models-based ensemble approach to driver drowsiness detection," Neural Computing & Applications, vol. 33, (8), pp. 3155-3168, Apr, 2021. DOI: 10.1007/s00521-020-05209- 7.   DOI
11 D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, vol. 60, (2), pp. 91-110, 2004.   DOI
12 D. Scherer, A. Muller and S. Behnke, "Evaluation of pooling operations in convolutional architectures for object recognition," in International Conference on Artificial Neural Networks, 2010, pp. 92-101.
13 H.J. Kim, H.J. Lim and J.H. Yang, "Design and Evaluation of Alert Threshold for Takeover Request in Partial Autonomous Vehicles considering Human Factors," in Proceedings of Conference, Korean Society of Automotive Engineers, 2016.
14 SAE International Releases Updated Visual Chart for Its "Levels of Driving Automation" Standard for Self-Driving Vehicles, https://www.sae.org/news/press-room/2018/12/sae-int ernational-releases-updated-visual-chart-for-its-%E2%80%9Clevels-of-driving-automation%E2%80%9D-standard-for-self-driving-vehicles.
15 E. Pakdamanian, S. Sheng, S. Baee, S. Heo, S. Kraus and L. Feng, "Deeptake: Prediction of driver takeover behavior using multimodal data," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-14, 2021.
16 Waymo. "Waymo Open Dataset." Available: https://waymo.com/open/data/.
17 Weconomy,. Self-Driving Vehicle Hits Bicyclist. Available: http://www.hani.co.kr/arti/economy/it/838665.html.
18 H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, "Speeded-up robust features (SURF)," Comput. Vision Image Understanding, vol. 110, (3), pp. 346-359, 2008.   DOI
19 K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
20 N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," The Journal of Machine Learning Research, vol. 15, (1), pp. 1929-1958, 2014.
21 L. Li, B. Zhong, C. Hutmacher, Y. Liang, W. J. Horrey and X. Xu, "Detection of driver manual distraction via image-based hand and ear recognition," Accident Analysis and Prevention, vol. 137, pp. 105432, Mar, 2020. DOI: 10.1016/j.aap.2020.105432.   DOI
22 S. Lee and S. Lee, "Moderating effect of speed desire frustration on the relationship between driving stress and coping behavior on traffic accident risk," Korean Journal of Industrial and Organizational Psychology, vol. 25, (3), pp. 571-587, 2012.   DOI
23 C. A. Pickering, K. J. Burnham, and M. J. Richardson, "A research study of hand gesture recognition technologies and applications for human vehicle interaction," Proceeding of 3rd Institution of Engineering and Technology conference on Automotive Electronics, pp. 1-15, Nov, 2007.
24 Z. Zhao, S. Xia, X. Xu, L. Zhang, H. Yan, Y. Xu and Z. Zhang, "Driver Distraction Detection Method Based on Continuous Head Pose Estimation," Computational Intelligence and Neuroscience, vol. 2020, Nov 29, 2020. DOI: 10.1155/2020/9606908.   DOI
25 A. Lobo, S. Ferreira and A. Couto, "Exploring Monitoring Systems Data for Driver Distraction and Drowsiness Research," Sensors (Basel, Switzerland), vol. 20, (14), pp. 3836, Jul 9, 2020. DOI: 10.3390/s20143836.   DOI
26 C. V. Hari and P. Sankaran, "Driver distraction analysis using face pose cues," Expert Systems with Applications, vol. 179, pp. 115036, Oct 1, 2021. DOI: 10.1016/j.eswa.202 1.115036.   DOI
27 F. Vicente, Zehua Huang, Xuehan Xiong, F. De la Torre, Wende Zhang and D. Levi, "Driver Gaze Tracking and Eyes Off the Road Detection System," IEEE Transactions on Intelligent Transportation Systems, vol. 16, (4), pp. 2014-2027, Aug, 2015. DOI: 10.1109/TITS.2015.2396031.   DOI
28 A. Bhatt, V. Dave, Y. Panchamia and P. Thakre, "Analyzing behavioral attributes of drivers and implementing safe driving model," in 2017 IEEE International Conference on Vehicular Electronics and Safety (ICVES), 2017, pp. 228-232.
29 H. J. Chung, "A Study on Effect of Women Driver's Dangerous Behavior based on the Theory of Planned Behavior(TPB)," Doctoral Thesis, Myongji University, Korea, 2013.
30 A. Koesdwiady, R. Soua, F. Karray and M. S. Kamel, "Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges," IEEE Transactions on Vehicular Technology, vol. 66, (6), pp. 4550-4563, Jun, 2017. DOI: 10.1109/TVT.2016.2631604.   DOI
31 B. Wagner, F. Taffner, S. Karaca and L. Karge, "Vision Based Detection of Driver Cell Phone Usage and Food Consumption," IEEE Transactions on Intelligent Transportation Systems, 2021.
32 A. A. Suzen, B. Duman and B. Şen, "Benchmark analysis of jetson TX2, jetson nano and raspberry PI using deep-CNN," in - 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), 2020, pp. 1-5. DOI: 10.1109/HORA4 9412.2020.9152915.   DOI
33 M. Raja, V. Ghaderi and S. Sigg, "WiBot! in-vehicle behaviour and gesture recognition using wireless network edge," Proceeding of IEEE 38th International Conference on Distributed Computing Systems (ICDCS), in Jul 2018, pp. 376-387.
34 G. Li, W. Yan, S. Li, X. Qu, W. Chu and D. Cao, "A Temporal-Spatial Deep Learning Approach for Driver Distraction Detection Based on EEG Signals," IEEE Transactions on Automation Science and Engineering, pp. 1-13, Jun 23, 2021. DOI: 10.1109/TASE.2021.3088897.   DOI
35 M. Karthaus, E. Wascher and S. Getzmann, "Distraction in the Driving Simulator: An Event-Related Potential (ERP) Study with Young, Middle-Aged, and Older Drivers," Safety (Basel), vol. 7, (2), pp. 36, May 11, 2021. DOI: 10.3390/safety7020036.   DOI
36 S. D. Ki, "An analysis on the effects of the driver distraction on the car accidents," Insurance and Financial Research, vol. 22, (3), pp. 3-32, 2011.
37 N. Arbabzadeh and M. Jafari, "A Data-Driven Approach for Driving Safety Risk Prediction Using Driver Behavior and Roadway Information Data," IEEE transactions on intelligent transportation systems, vol. 19, (2), pp. 446-460, Feb, 2018. DOI: 10.1109/TITS.2017.2700869.   DOI
38 G. H. Choi, K. Lim and S. B. Pan, "Driver Identification System Using Normalized Electrocardiogram Based on Adaptive Threshold Filter for Intelligent Vehicles," Sensors (Basel), vol. 21, (1), pp. 10.3390/s21010202, December 30, 2020. . DOI: https://doi.org/10.3390/s21010202.   DOI
39 Y. Xing, C. Lv, Z. Zhang, H. Wang, X. Na, D. Cao, E. Velenis and F. Y. Wang, "Identification and analysis of driver postures for in-vehicle driving activities and secondary tasks recognition," IEEE Transactions on Computational Social Systems, Vol. 5, No. 1, pp. 95-108, March, 2018.   DOI
40 M. Mori, C. Miyajima, P. Angkititrakul, T. Hirayama, Y. Li, N. Kitaoka, and K. Takeda, "Measuring driver awareness based on correlation between gaze behavior and risks of surrounding vehicles," Proceeding of 2012 15th International IEEE Conference on Intelligent Transportation Systems, in Sep 2012, pp. 644-647.
41 Y. Zhang, Y. Chen and C. Gao, "Deep unsupervised multi-modal fusion network for detecting driver distraction," Neurocomputing (Amsterdam), vol. 421, pp. 26-38, Jan 15, 2021. DOI: 10.1016/j.neucom.2020.09.023.   DOI
42 Ministry of Land, Monthly vehicle registration statistics. Available: www.molt.go.kr.
43 National Highway Traffic Safety Administration, "Preliminary statement of policy concerning automated vehicles. washington, dc",. Available: www.nhtsa.gov/staticfiles/rulema king/pdf/Automated_Vehicles_Policy.pdf.
44 A. Kashevnik, R. Shchedrin, C. Kaiser and A. Stocker, "Driver Distraction Detection Methods: A Literature Review and Framework," in IEEE Access, vol. 9, pp. 60063-60076, 2021, DOI: 10.1109/ACCESS.2021.3073599.   DOI
45 M. Costa, D. Oliveira, S. Pinto and A. Tavares, "Detecting Driver's Fatigue, Distraction and Activity Using a Non-Intrusive Ai-Based Monitoring System," Journal of Artificial Intelligence and Soft Computing Research, vol. 9, (4), pp. 247-266, Oct 1, 2019. DOI: 10.2478/jaiscr-2019-00 07.   DOI
46 N. J. Dunn, T. A. Dingus, S. Soccolich and W. J. Horrey, "Investigating the impact of driving automation systems on distracted driving behaviors," Accident Analysis & Prevention, vol. 156, pp. 106152, 2021. DOI: 10.1016/j.aap. 2021.106152.   DOI
47 Kaggle. State Farm Distracted Driver Detection. Available: https://www.kaggle.com/c/state-farm-distracted-driver-detection/
48 B. Zhou, A. Khosla, A. Lapedriza, A. Oliva and A. Torralba, "Learning deep features for discriminative localization," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2921-2929.
49 Wencai Sun, Yihao Si, Mengzhu Guo and Shiwu Li, "Driver Distraction Recognition Using Wearable IMU Sensor Data," Sustainability (Basel, Switzerland), vol. 13, (1342), pp. 1342, Jan 1, 2021. DOI: 10.3390/su13031342.   DOI