Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Predicting Driver Readiness in Conditional Autonomous Driving: Analyzing Driver Characteristics and Heart Rate Variability

조건부 자율주행 동안 운전자의 주관적인 운전 준비도 예측: 운전자 특성과 심박변이도 분석

  • 장은혜 (한국전자통신연구원 모빌리티UX연구실 ) ;
  • 강도욱 (한국전자통신연구원 모빌리티UX연구실) ;
  • 김우진 (한국전자통신연구원 모빌리티UX연구실) ;
  • 윤대섭 (한국전자통신연구원 모빌리티UX연구실) ;
  • 장미 (한국전자통신연구원 모빌리티UX연구실)
  • Received : 2024.09.13
  • Accepted : 2024.11.29
  • Published : 2024.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study investigates psychological and physiological indicators to measure drivers' subjective readiness in autonomous driving situations objectively. A total of 51 participants participated in the study, and pre-experimental evaluations employed surveys to assess driving experience, attitude, driving workload, and situation awareness (SA). This study utilized heart rate variability as a physiological indicator, with electrocardiogram data collected during autonomous driving simulations, particularly during driving scenarios requiring drivers' manual control interventions. Afterward, the participants assessed their state while driving. Results revealed a negative correlation between drivers' readiness and mental demand and a positive correlation with SA and understanding of the situation. Moreover, results demonstrated significant positive correlations between drivers' readiness and heart rate variability metrics, particularly the square root of the mean of the sum of the square of differences between adjacent NN intervals (RMSSD) and the proportion-derived NN50 by the total number of NN intervals (pNN50). Comparative analysis of high, medium, and low readiness groups presented significant disparities. The high-readiness group exhibited lower mental demand, higher SA and understanding of the situation, and higher pNN50 during the autonomous driving phase before the takeover. Finally, SA and RMSSD were identified as key predictors of driver readiness. Drivers with lower readiness exhibited heightened autonomic arousal, while those with higher readiness displayed enhanced psychological and physiological stability, attributed to increased parasympathetic activation. These findings highlight the significance of psychological and physiological indicators in predicting readiness, offering valuable insights for developing monitoring technologies to improve driver safety and accident prevention.

본 연구는 자율주행상황에서 주관적인 운전 준비도를 객관적으로 측정할 수 있는 심리·생리적 지표를 확인하는 것을 목적으로 한다. 51명의 연구대상자가 참여하였고, 설문을 통해 운전 경험, 태도, 운전부하, 상황인식 등을 평가하였다. 자율주행 중 차량 제어권을 인계받아야 하는 시나리오 동안 심전도를 측정하여 심박변이도 지표를 추출하였고, 주행 종료 후 연구대상자는 자신의 상태를 평가하였다. 분석 결과, 운전 준비도는 정신적 부하와 부적 상관, 상황 인식과 상황 이해도와는 정적 상관을 보였다. 또한, 심박변이도 지표인 제곱 평균 근간 심박 간격 차이(Root Mean Square of Successive Differences, RMSSD)와 50ms 이상의 연속적인 RR 간격의 차이 비율(proportion derived NN50 by the total number of NN interval, pNN50)과의 유의한 정적 상관관계가 확인되었다. 운전 준비도 수준에 따라 상·중·하로 나누어 분석한 결과, 높은 운전 준비도 집단은 정신적 부하가 낮고 상황인식 및 상황에 대한 이해도가 유의하게 높았으며, 자율주행 구간에서 pNN50이 높은 경향이 있었다. 마지막으로 상황인식과 RMSSD가 운전 준비도의 주요 예측 지표로 확인되었다. 이는 운전 준비도가 낮은 운전자는 자율신경 각성이 높고, 높은 운전자는 부교감신경계의 활성화로 인해 심리적, 생리적으로 안정된 상태임을 의미한다. 본 연구는 운전자의 주관적인 운전 준비도를 예측하기 위한 운전자의 심리 및 생리 지표를 확인하였고, 이는 운전자의 운전 준비 상태를 모니터링하는 기술에 적용되어 사고 예방에 기여할 수 있을 것이다.

Keywords

Acknowledgement

본 연구는 산업통상자원부/한국산업기술평가관리원의 지원으로 수행되었음(No. 20024892, 운전자 신체적·인지적 상태 판단 이종센서융합 모듈 및 통합시스템 개발).

References

  1. Aberg, L., & Rimmo, P. A. (1998). Dimensions of aberrant driver behaviour. Ergonomics, 41(1), 39-56. DOI: 10.1080/001401398187314.
  2. Arslanyilmaz, A., Matouq, S., & Doner, D. (2020). Driver readiness in autonomous vehicle take-overs. International Journal of Transport and Vehicle Engineering, 14(8), 558-563. DOI: 10.3390/safety6010012.
  3. Berntson, G. G., Bigger, J. T. J., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., Nagaraja, H. N., Porges, S. W., Saul, J. P., Stone, P. H., & van der Molen, M. W. (1997). Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology, 34(6), 623-648. DOI: 10.1111/j.1469-8986.1997.tb02140.x.
  4. Bibbo, D., Mariajoseph, M., Gallina, B., & Carli, M. (2022). A novel physiological-based system to assess drivers' stress during earth moving simulated activities. Electronics, 11(24), 4074. DOI: 10.3390/electronics11244074.
  5. Bolton, M. L., Biltekoff, E., Wei, J., & Humphrey, L. (2022). On the level of measurement of subjective psychometric ratings. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 66(1), 80-84. Sage CA: Los Angeles, CA: SAGE Publications. DOI: 10.1177/107118132266.
  6. Braithwaite, J. J., Watson, D. G., Jones, R., & Rowe, M. (2013). A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology, 49(1), 1017-1034.
  7. Braunagel, C., Rosenstiel, W., & Kasneci, E. (2017). Ready for take-over? A new driver assistance system for an automated classification of driver take-over readiness. IEEE Intelligent Transportation Systems Magazine, 9(4), 10-22. DOI: 10.1109/MITS.2017. 2743165.
  8. Carsten, O. (2007). From driver models to modelling the driver: what do we really need to know about the driver? In: Cacciabue, P.C. (Ed.), Modelling Driver Behaviour in Automotive Environments. Springer, London, 105-120.
  9. da Silva, F. P. (2014). Mental workload, task demand and driving performance: what relation?. Procedia-Social and Behavioral Sciences, 162, 310-319. DOI: 10.1016/j.sbspro.2014.12.212.
  10. Deng, M., Gluck, A., Zhao, Y., Li, D., Menassa, C. C., Kamat, V. R., & Brinkley, J. (2024). An analysis of physiological responses as indicators of driver takeover readiness in conditionally automated driving. Accident Analysis & Prevention, 195, 107372. DOI: 10.1016/j.aap.2023.107372.
  11. Deo, N., & Trivedi, M. M. (2019). Looking at the driver/rider in autonomous vehicles to predict take-over readiness. IEEE Transactions on Intelligent Vehicles, 5(1), 41-52. DOI: 10.1109/TIV.2019.2955364.
  12. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017). CARLA: An open urban driving simulator. In Conference on robot learning (pp. 1-16). PMLR. DOI: 10.48550/arXiv.1711.03938.
  13. Endsley, M. R. (1989). Final report: Situation awareness in an advanced strategic mission (NOR DOC 89-32). Hawthorne, CA: Northrop Corporation.
  14. Fujiwara, K., Abe, E., Kamata, K., Nakayama, C., Suzuki, Y., Yamakawa, T., Hiraoka, T., Kano, M., Sumi, Y., Masuda, F., Matsuo, M., & Kadotani, H. (2019). Heart rate variability-based driver drowsiness detection and its validation with EEG. IEEE Transactions on Biomedical Engineering, 66(6), 1769-1778. DOI: 10.1109/TBME.2018.2879346.
  15. Gonçalves, R. C., Goodridge, C. M., Kuo, J., Lenné, M. G., & Merat, N. (2024). Using driver monitoring to estimate readiness in automation: a conceptual model based on simulator experimental data. Cogn Tech Work. DOI: 10.1007/s10111-024-00777-3.
  16. Gravetter, F. J., & Wallnau, L. B. (2013). Statistics for the Behavioral Sciences (9th ed.). Cengage Learning.
  17. Greer, R., Deo, N., Rangesh, A., Gunaratne, P., & Trivedi, M. (2023). Safe control transitions: Machine vision based observable readiness index and data-driven takeover time prediction. arXiv preprint arXiv:2301.05805. DOI: 10.48550/arXiv.2301.05805.
  18. Guo, W., Tian, X., Tan, J., & Wang, L. (2016). Change in heart rate variability indexes due to high driving workload in turning left at the intersection in real road environment. In 2016 3rd International Conference on Materials Engineering, Manufacturing Technology and Control (pp. 804-807). Atlantis Press. DOI: 10.2991/icmemtc-16.2016.158.
  19. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, Advances in Psychology 52, 139-83. DOI: 10.1016/S0166-4115(08)62386-9.
  20. Hatakka, M., Keskinen, E., Gregersen, N. P., Glad, A., & Hernetkoski, K. (2002). From control of the vehicle to personal self-control; broadening the perspectives to driver education. Transportation Research Part F: Traffic Psychology and Behaviour, 5(3), 201-215. DOI: 110.1016/S1369-8478(02)00018-9.
  21. Kantowitz, B. H., & Simsek, O. (2000). Secondary-task measures of driver workload. In Stress, workload, and fatigue (pp. 395-408). CRC Press. DOI: 10.1201/b12791-2.10.
  22. Kass, S. J., Cole, K. S., & Stanny, C. J. (2007). Effects of distraction and experience on situation awareness and simulated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 10(4), 321-329. DOI: 10.1016/j.trf.2006.12.002.
  23. Kazemi, M., Rezaei, M., & Azarmi, M. (2024). Evaluating Driver Readiness in Conditionally Automated Vehicles from Eye-Tracking Data and Head Pose. ArXiv, abs/2401.11284. DOI: 10.48550/arXiv.2401.11284.
  24. Kim H.S., Kwon O.C., Lee S. J., Kim J. S., Kim W. J., Yoon D. S., & Lee I. H. (2020). A Study on Human Factors Guidelines for Level 3 Automated Vehicles. Electronics and Telecommunications Trends. 35(6), 24-36. DOI: 10.22648/ETRI.2020.J.350603.
  25. Kim, J., Kim, W., Kim, H. S., Lee, S. J., Kwon, O. C., & Yoon, D. (2022). A novel study on subjective driver readiness in terms of non-driving related tasks and take-over performance. ICT Express, 8(1), 91-96. DOI: 10.1109/ICTC49870.2020.9289293.
  26. Koessmeier, C., & Büttner, O. B. (2022). Beyond the Smartphone's mere presence effect: a quantitative mobile eye tracking study on the visual and internal distraction potential of smartphones. Computers in Human Behavior, 134, 107333. DOI: 10.1016/j.chb.2022.107333.
  27. Lawton, R., Parker, D., Manstead, A. S., & Stradling, S. G. (1997). The role of affect in predicting social behaviors: the case of road traffic violations. Journal of Applied Social Psychology, 27(14), 1258-1276. DOI: 10.1111/j.1559-1816.1997.tb01805.x.
  28. Lee, Y. H., Jeon, J. -D. and Choi, Y. -C. (2012), Air traffic controllers' situation awareness and workload under dynamic air traffic situations. Transportation Journal, 51, 338-352. DOI: 10.5325/transportationj.51.3.0338.
  29. Lu, K., Sjörs Dahlman, A., Karlsson, J., & Candefjord, S. (2022). Detecting driver fatigue using heart rate variability: a systematic review. Accident Analysis and Prevention, 178, 1-10. DOI: 10.1016/j.aap.2022.106830.
  30. Lu, Z., Happee, R., Cabrall, C. D., Kyriakidis, M., & De Winter, J. C. (2016). Human factors of transitions in automated driving: a general framework and literature survey. Transportation Research Part F: Traffic Psychology and Behaviour, 43, 183-198. DOI: 10.1016/j.trf.2016.10.007.
  31. Martinussen, L. M., Lajunen, T., Møller, M., & Özkan, T. (2013). Short and user-friendly: The development and validation of the Mini-DBQ. Accident Analysis & Prevention, 50, 1259-1265. DOI: 10.1016/j.aap.2012.09.030.
  32. McDonald, A. D., Alambeigi, H., Engström, J., Markkula, G., Vogelpohl, T., Dunne, J., & Yuma, N. (2019). Toward computational simulations of behavior during automated driving takeovers: a review of the empirical and modeling literatures. Human Factors, 61(4), 642-688. DOI: 10.1177/0018720819829572.
  33. Meteier, Q., Capallera, M., Ruffieux, S., Angelini, L., Abou Khaled, O., Mugellini, E., Carrino, S., & Sonderegger, A. (2021). Classification of drivers' workload using physiological signals in conditional automation. Frontiers in Psychology, 12, 596038. DOI: 10.3389/fpsyg.2021.596038.
  34. Michon, J. A. (1985). A critical view of driver behavior models: what do we know, what should we do?. In: Evans, L., Schwing, R.C. (eds) Human behavior and traffic safety. Springer, Boston, MA. DOI: 10.1007/978-1-4613-2173-6_19.
  35. Morando, A., Victor, T., Bengler, K., & Dozza, M. (2020). Users' response to critical situations in automated driving: rear-ends, sideswipes, and false warnings. IEEE Transactions on Intelligent Transportation Systems, 22(5), 2809-2822. DOI: 10.1109/TITS.2020.2975429.
  36. Mukaka, M. M. (2012). Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Medical Journal, 24(3), 69-71.
  37. Nasoz, F., Alvarez, K., Lisetti, C. L., & Finkelstein, N. (2004). Emotion recognition from physiological signals using wireless sensors for presence technologies. Cognition, Technology & Work, 6, 4-14. DOI:10.1007/s10111-003-0143-x.
  38. Naujoks, F., Höfling, S., Purucker, C., & Zeeb, K. (2018). From partial and high automation to manual driving: Relationship between non-driving related tasks, drowsiness and take-over performance. Accident Analysis & Prevention, 121, 28-42. DOI: 10.1016/j.aap.2018.08.018.
  39. Pakdamanian, E., Sheng, S., Baee, S., Heo, S., Kraus, S., & Feng, L. (2021). Deeptake: Prediction of driver takeover behavior using multimodal data. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-14). DOI: 10.1145/3411764.344556.
  40. Parker, D., Reason, J. T., Manstead, A. S., & Stradling, S. G. (1995). Driving errors, driving violations and accident involvement. Ergonomics, 38(5), 1036-1048. DOI: 10.1080/00140139508925170.
  41. Pipkorn, L., Tivesten, E., & Dozza, M. (2022). It's about time! Earlier take-over requests in automated driving enable safer responses to conflicts. Transportation Research Part F: Traffic Psychology and Behaviour, 86, 196-209. DOI: 10.1016/j.trf.2022.02.014.
  42. Rimmö, P. A. (2002). Aberrant driving behaviour: homogeneity of a four-factor structure in samples differing in age and gender. Ergonomics, 45(8), 569-582. DOI: 10.1080/00140130210145873.
  43. Rostamzadeh, S., Abouhossein, A., Vosoughi, S., Gendeshmin, S. B., & Yarahmadi, R. (2024). Stress influence on real-world driving identified by monitoring heart rate variability and morphologic variability of electrocardiogram signals: the case of intercity roads. International Journal of Occupational Safety and Ergonomics, 30(1), 252-263. DOI: 10.1080/10803548.2023.2293391.
  44. Ryu, J. B., & Oh, J. S. (2022). Development of Korean version of Driving behavior scale. Traffic Safety Research, 41(1), 47-64. DOI: 10.22940/TSR.2022.41.1.47.
  45. Sugiono, S., Widhayanuriyawan, D., & Andriani, D. P. (2017). Investigating the impact of road condition complexity on driving workload. MATEC Web of Conferences 136, 02007. DOI: 10.1051/matecconf/201713602007.
  46. Taylor, R. M. (1990). Situation awareness rating technique (SART): the development of a tool for aircrew systems design. In Situational Awareness in Aerospace Operations (Chapter 3). France: Neuilly sur-Seine, NATO-AGARD-CP-478.
  47. Teshima, T., Niitsuma, M., & Nishimura, H. (2024). Determining the onset of driver's preparatory action for take-over in automated driving using multimodal data. Expert Systems with Applications, 246, 123153. DOI: 10.1016/j.eswa.2024.123153.
  48. Verwey, W. B. (2000). On-line driver workload estimation. Effects of road situation and age on secondary task measures. Ergonomics, 43(2), 187-209. DOI: 10.1080/001401300184558.
  49. Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous driving: investigating the feasibility of car-driver handover assistance. In Proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications (pp. 11-18). DOI: 10.1145/2799250.2799268.
  50. Yang, Y., Ye, Z., Easa, S. M., Feng, Y., & Zheng, X. (2023). Effect of driving distractions on driver mental workload in work zone's warning area. Transportation Research Part F: Traffic Psychology and Behaviour, 95, 112-128. DOI: 10.1016/j.trf.2023.03.018.
  51. Yu, Z., Xu, G., Jiang, K., Feng, Z., & Xu, S. (2023). Constructing the behavioral sequence of the takeover process—TOR, behavior characteristics and phases division: a real vehicle experiment. Accident Analysis & Prevention, 186, 107040. DOI: 10.1016/j.aap.2023.107040.
  52. Zeeb, K., Buchner, A., & Schrauf, M. (2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212-221. DOI: 10.1016/j.aap.2015.02.023.
  53. Zeng, C., Zhang, J., Su, Y., Li, S., Wang, Z., Li, Q., & Wang, W. (2024). Driver fatigue detection using heart rate variability features from 2-minute electrocardiogram signals while accounting for sex differences. Sensors, 24(13), 4316. DOI: 10.3390/s24134316.