ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Multimodal Interface Based on Novel HMI UI/UX for In-Vehicle Infotainment System

  • Kim, Jinwoo (IT Convergence Technology Research Laboratory, ETRI) ;
  • Ryu, Jae Hong (IT Convergence Technology Research Laboratory, ETRI) ;
  • Han, Tae Man (IT Convergence Technology Research Laboratory, ETRI)
  • Received : 2014.08.10
  • Accepted : 2015.05.14
  • Published : 2015.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a novel HMI UI/UX for an in-vehicle infotainment system. Our proposed HMI UI comprises multimodal interfaces that allow a driver to safely and intuitively manipulate an infotainment system while driving. Our analysis of a touchscreen interface-based HMI UI/UX reveals that a driver's use of such an interface while driving can cause the driver to be seriously distracted. Our proposed HMI UI/UX is a novel manipulation mechanism for a vehicle infotainment service. It consists of several interfaces that incorporate a variety of modalities, such as speech recognition, a manipulating device, and hand gesture recognition. In addition, we provide an HMI UI framework designed to be manipulated using a simple method based on four directions and one selection motion. Extensive quantitative and qualitative in-vehicle experiments demonstrate that the proposed HMI UI/UX is an efficient mechanism through which to manipulate an infotainment system while driving.

Keywords

References

  1. H.M. Kim et al., "Dual Autostereoscopic Display Platform for Multi-user Collaboration with Natural Interaction," ETRI J., vol. 34, no. 3, June 2012, pp. 466-469. https://doi.org/10.4218/etrij.12.0211.0331
  2. D. Lee and S. Lee, "Vision-Based Finger Action Recognition by Angle Detection and Contour Analysis," ETRI J., vol. 33, no. 3, June 2011, pp. 415-422. https://doi.org/10.4218/etrij.11.0110.0313
  3. L. Hackenberg et al., "International Evaluation of NLU Benefits in the Domain of In-Vehicle Speech Dialog Systems," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 114-120.
  4. NHTSA, "The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data," National Highway Safety Administration, Washington, DC, USA, Rep. DOT HS 810 594, 2006.
  5. T.M. Gable et al., "Advanced Auditory Cues on Mobile Phones Help Keep Drivers' Eyes on the Road," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 66-73.
  6. T. Kujala, J. Silvennoinen, and A. Lasch, "Visual-Manual In-Car Tasks Decomposed - Text Entry and Kinetic Scrolling as the Main Sources of Visual Distraction," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 82-89.
  7. I. Politis, S. Brewster, and F. Pollick, "Evaluating Multimodal Driver Displays of Varying Urgency," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 92-99.
  8. G. Burnett et al., "A Study of Unidirectional Swipe Gestures on In-Vehicle Touch Screens," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 22-29.
  9. A. Riener et al., "Standardization of the In-Car Gesture Interaction Space," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Eindhoven, Netherlands, Oct. 28-30, 2013, pp. 14-21.
  10. C. Jablonski, "An Augmented Reality Windshield from GM," Mar. 19, 2010. Accessed Jan. 20, 2015. http://www.zdnet.com/article/anaugmented-reality-windshield-from-gm/
  11. R. Ecker et al., "Visual Cues Supporting Direct Touch Gesture Interaction with In-Vehicle Information Systems," Proc. Int. Conf. Automotive User Interfaces Interactive Veh. Appl., Pittsburgh, PA, USA, Nov. 11-12, 2010, pp. 80-87.
  12. M.A. Perez, "Safety Implications of Infotainment System Use in Naturalistic Driving," J. Prevention, Assessment, Rehabil., vol. 41, Feb. 2012, pp. 4200-4204.
  13. H.S. Park et al., "In-Vehicle AR-HUD System to Provide Driving-Safety Information," ETRI J., vol. 35, no. 6, Dec. 2013, pp. 1038-1047. https://doi.org/10.4218/etrij.13.2013.0041
  14. N. Parimal et al., "Application of Sensors in Augmented Reality Based Interactive Learning Environments," IEEE Int. Conf. Sens. Technol., Kolkata, India, Dec. 18-21, 2012, pp. 173-178.
  15. S. Khan, "SVP Infotainment and Connected Drive," BMW Case Study, BMW Group Corp., 2008. Accessed May. 10, 2014. http://www.genivi.org/sites/default/files/BMW_Case_Study_Download_040914.pdf
  16. Developer Center, GestureList, Leapmotion Corp., 2014. Accessed Dec. 14, 2014. https://developer.leapmotion.com/documentation/skeletal/cpp/api/Leap.Gesture.html
  17. Naver Developer Center, Map API, Naver Corp., 2014. Accessed Dec. 14, 2014. http://developer.naver.com/wiki/pages/JavaScript
  18. 3D Connexion Support team, Space Navigator for Notebooks, 3D Connextion Corp., 2014. Accessed Nov. 14, 2014. http://www.3dconnexion.eu/products/spacenavigator-for-notebooks.html
  19. SCS Software, Euro Truck Simulator 2, SCS Software Corp., Accessed Mar. 17, 2015. http://www.eurotrucksimulator2.com/world.php
  20. S. Franzen and M. Babapour, "HMI Certification - a Critical Review of Methods for Safety and Deficiency Evaluation of HMI Solutions for IVIS," Chalmers University of Technology, Dept. Des. & Human Factors, July 2011.
  21. P. Seewald et al., "D13.2: Results of HMI and Feedback Solutions Evaluations," EcoDriver, SP1: Supporting Drivers in Eco-driving, Tech. Rep. D13.2, Ver. 10, Nov. 2013.

Cited by

  1. Effects of Augmented-Reality Head-up Display System Use on Risk Perception and Psychological Changes of Drivers vol.38, pp.4, 2015, https://doi.org/10.4218/etrij.16.0115.0770
  2. A real-time gesture recognition system using near-infrared imagery vol.14, pp.10, 2019, https://doi.org/10.1371/journal.pone.0223320
  3. OpenCL-Darknet: implementation and optimization of OpenCL-based deep learning object detection framework vol.24, pp.4, 2021, https://doi.org/10.1007/s11280-020-00778-y
  4. Effect of Touch Button Interface on In-Vehicle Information Systems Usability vol.37, pp.15, 2021, https://doi.org/10.1080/10447318.2021.1886484
  5. Analysis of Relationship between Electroencephalograms and Subjective Measurements for In-Vehicle Information System: A Preliminary Study vol.18, pp.22, 2015, https://doi.org/10.3390/ijerph182212173