ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

In-Vehicle AR-HUD System to Provide Driving-Safety Information

  • Park, Hye Sun (IT Convergence Technology Research Laboratory, ETRI) ;
  • Park, Min Woo (Virtual Reality Laboratory, Kyungpook National University) ;
  • Won, Kwang Hee (Virtual Reality Laboratory, Kyungpook National University) ;
  • Kim, Kyong-Ho (IT Convergence Technology Research Laboratory, ETRI) ;
  • Jung, Soon Ki (Virtual Reality Laboratory, Kyungpook National University)
  • Received : 2013.03.31
  • Accepted : 2013.10.26
  • Published : 2013.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Augmented reality (AR) is currently being applied actively to commercial products, and various types of intelligent AR systems combining both the Global Positioning System and computer-vision technologies are being developed and commercialized. This paper suggests an in-vehicle head-up display (HUD) system that is combined with AR technology. The proposed system recognizes driving-safety information and offers it to the driver. Unlike existing HUD systems, the system displays information registered to the driver's view and is developed for the robust recognition of obstacles under bad weather conditions. The system is composed of four modules: a ground obstacle detection module, an object decision module, an object recognition module, and a display module. The recognition ratio of the driving-safety information obtained by the proposed AR-HUD system is about 73%, and the system has a recognition speed of about 15 fps for both vehicles and pedestrians.

Keywords

References

  1. S.H. Lee, J. Choi, and J. Park, "Interactive e-Learning System Using Pattern Recognition and Augmented Reality," IEEE Trans. Consum. Electron., vol. 55, no. 2, May 2009, pp. 883-890. https://doi.org/10.1109/TCE.2009.5174470
  2. B. Choi et al., "A Metadata Design for Augmented Broadcasting and Testbed System Implementation," ETRI J., vol. 35, no. 2, Apr. 2013, pp. 292-300. https://doi.org/10.4218/etrij.13.0112.0412
  3. N. Parimal et al., "Application of Sensors in Augmented Reality Based Interactive Learning Environments," IEEE Int. Conf. ICST, Dec. 2012, pp. 173-178.
  4. M.A. Oikawa et al., "AR-Based Video-Mediated Communication: A Social Presence Enhancing Experience," 14th Int. Symp. VR, May. 2012, pp. 125-130.
  5. Y.M. Aung and A. Al-Jumaily, "AR Based Upper Limb Rehabilitation System," IEEE Int. Conf. BioRob., June 2012, pp. 213-218.
  6. T. Zysk, J. Luce, and J. Cunningham, "Augmented Reality for Precision Navigation," GPS World Mag., vol. 23, issue 6, June 2012, pp. 47-51.
  7. H. Chen et al., "Application of Augmented Reality in Engineering Graphics Education," IEEE Int. Symp. ITME 2, art. no. 6132125, Dec. 2011, pp. 362-365.
  8. Y.C. Cheng et al., "AR-Based Positioning for Mobile Devices," Int. Conf. Parallel Process., no. 6047276, Sept. 2011, pp. 63-70.
  9. J.J. Mccorvey, "Using Augmented Reality to Market Your Business," Inc., Apr. 2010. http://www.inc.com/magazine/ 20100401/using-augmented-reality-to-market-your-business.html
  10. Siemens, "From Sedans to Compact Cars - One HUD Fits All," press photo, reference no. 200605004, Siemens AG, Munich, Germany, 2006. http://www.siemens.com/press/en/presspicture/? press=/en/pp_sv/2006/sv200605004_(hud)_1376467.htm
  11. J. Lincoln, D. Lincoln, and M. Carmean, How a Laser HUD Can Make Driving Safer, MicroVision Program Brief, MicroVision, Inc., Mar. 2007.
  12. R. Parasuraman, R. Molloy, and I.L. Singh, "Performance Consequences of Automation-Induced Complacency," Int. J. Aviation Psychology, vol. 3, no. 1, 1993, pp. 1-23. https://doi.org/10.1207/s15327108ijap0301_1
  13. NHTSA, Measuring Distraction Potential of Operating In-Vehicle Devices, no. DOT-HS-811-231, Dec. 2008.
  14. S.C. Lee, "Psychological Effects on Aged Driver's Traffic Accidents," Korea J. Psychological Soc. Issues, vol. 12, 2006, pp. 149-167.
  15. Y. Zhang, Y. Owechko, and J. Zhang, "Learning-Based Driver Workload Estimation," Comput. Intell. Automobile Appl., Ser. Studies Comput. Intell., vol. 132, 2008, pp. 1-24.
  16. E.R. Michael, J.G. Leo, and J.W. Nicholas, "Effects of Naturalistic Cell Phone Conversations on Driving Performance," Int. J. Safety Research, vol. 35, 2004, pp. 453-464. https://doi.org/10.1016/j.jsr.2004.06.003
  17. E. Adella, A. Varhely, and M. Fontana, "The Effects of a Driver Assistance System for Safe Speed and Safe Distance: A Real-Life Field Study," Int. J. Transp. Research Part C 19, 2011, pp. 145-155. https://doi.org/10.1016/j.trc.2010.04.006
  18. E. Johan, J. Emma, and O. Joakim, "Effects of Visual and Cognitive Load in Real and Simulated Motorway Driving," Int. J. Transp. Research Part F, vol. 8, 2005, pp. 97-120. https://doi.org/10.1016/j.trf.2005.04.012
  19. R. Venkatasawmy, The Digitization of Cinematic Visual Effects: Hollywood's Coming of Age, UK: Press Lexington Books, 2013.
  20. A.S. Cohen and R. Hirsig, Feed Forward Programming of Car Driver's Eye Movement Behavior: A System Theoretical Approach, final technical report, vol. 2, 1980, pp. 1-223.
  21. V. Charissis and S. Papanastasiou, "Human-Machine Collaboration through Vehicle Head Up Display Interface," Int. J. Cognition, Technol., Work, vol. 12, no. 1, 2010, pp. 41-50. https://doi.org/10.1007/s10111-008-0117-0
  22. Y.C. Shinko, D. Anup, and M.T. Mohan, "Active Heads-up Display Based Speed Compliance Aid for Driver Assistance: A Novel Interface and Comparative Experimental Studies," IEEE Int. Symp. IV, 2007, pp. 594-599.
  23. A. Sato et al., "Visual Navigation System on Windshield Head-Up Display," Proc. 13th World Congress ITS, 2006, pp. 167-175.
  24. H. Watanabe et al., "The Effect of HUD Warning Location on Driver Responses," Proc. 6th World Congress ITS, 1999, pp. 1-10.
  25. C. Jablonski, "An Augmented Reality Windshield from GM," Mar. 2010. http://www.zdnet.com/blog/emergingtech/anaugmented- reality-windshield-from-gm/2164
  26. V. Poortere et al., "Efficient Pedestrian Detection: A Test Case for SVM Based Categorization," Proc. ICVW, Sept. 2002, pp. 19-20.
  27. D.M. Gavrila and V. Philomin, "Real-Time Object Detection for Smart Vehicles," IEEE Int. Conf. CVPR, 1999, pp. 87-93.
  28. S. Hermann and R. Klette, "Iterative Semi-Global Matching for Robust Driver Assistance Systems," Int. Conf. ACCV, vol. 7726, 2012, pp. 456-478.
  29. H. Hattori et al., "Stereo-Based Pedestrian Detection Using Multiple Patterns," Int. Conf. BMVC, 2009, pp. 1-10.
  30. M. Bajracharya et al., "A Fast Stereo-Based System for Detecting and Tracking Pedestrians from a Moving Vehicle," Int. J. Robotics Research, vol. 28, 2009, pp. 1466-1485. https://doi.org/10.1177/0278364909341884
  31. A. Shashua, Y. Gdalyahu, and G. Hayun, "Pedestrian Detection for Driving Assistance Systems: Single-Frame Classification and System Level Performance," IEEE Int. Symp. IV, 2004, pp. 1-6.
  32. S. Munder and D.M. Gavrila, "An Experimental Study on Pedestrian Classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, 2006, pp. 1863-1868. https://doi.org/10.1109/TPAMI.2006.217
  33. Australian Transport Council, The National Road Safety Strategy 2011-2020, 2011, pp. 1-108.
  34. K.H. Won and S.K. Jung, "Billboard Sweep Stereo for Obstacle Detection in Road Scenes," Int. J. IET, vol. 48, issue 24, Nov. 2012, pp. 1528-1530.
  35. M.W. Park, K.H. Won, and S.K. Jung, "Vehicle Detection and Tracking Using Billboard Sweep Stereo Matching Algorithm," Korea J. KMMS, vol. 16, no. 6, 2013, pp. 764-781.
  36. Z.H. Zhou and X. Geng, "Projection Functions for Eye Detection," Int. J. Pattern Recognition, vol. 37, no. 5, 2004, pp. 1049-1056. https://doi.org/10.1016/j.patcog.2003.09.006
  37. C. Cortes and V. Vapnik, "Support-Vector Networks," Int. J. Mach. Learning, vol. 20, no. 3, 1995, pp. 273-297.
  38. T. Joachims, "Making Large-Scale SVM Learning Practical," Advances in Kernel Methods - Support Vector Learning, B. Scholkopf, C. Burges, and A. Smola, Eds., Cambridge, MA: MIT Press, 1999.
  39. K.A. Paul and H. Ayman, "A Comparative Analysis between Rigorous and Approximate Approaches for LiDAR System Calibration," Korea J. KSGPC, vol. 30, no. 6, 2012, pp. 593-605.
  40. Wikipedia, Precision and recall. http://en.wikipedia.org/wiki/ Precision_and_recall
  41. Gamedev.net, Calculate framerate with GetTickCount function. http://www.gamedev.net/topic/621703-calculate-framerate-withgettickcount/

Cited by

  1. Robust Sign Recognition System at Subway Stations Using Verification Knowledge vol.36, pp.5, 2013, https://doi.org/10.4218/etrij.14.2214.0007
  2. Performance of vehicle speed estimation using wireless sensor networks: a region-based approach vol.71, pp.6, 2013, https://doi.org/10.1007/s11227-014-1306-7
  3. Multimodal Interface Based on Novel HMI UI/UX for In-Vehicle Infotainment System vol.37, pp.4, 2013, https://doi.org/10.4218/etrij.15.0114.0076
  4. Fixed Homography-Based Real-Time SW/HW Image Stitching Engine for Motor Vehicles vol.37, pp.6, 2013, https://doi.org/10.4218/etrij.15.0113.1120
  5. Analysis of Physiological Responses and Use of Fuzzy Information Granulation-Based Neural Network for Recognition of Three Emotions vol.37, pp.6, 2015, https://doi.org/10.4218/etrij.15.0114.0089
  6. Effects of Augmented-Reality Head-up Display System Use on Risk Perception and Psychological Changes of Drivers vol.38, pp.4, 2013, https://doi.org/10.4218/etrij.16.0115.0770
  7. Data-Driven Kinematic Control for Robotic Spatial Augmented Reality System with Loose Kinematic Specifications vol.38, pp.2, 2013, https://doi.org/10.4218/etrij.16.0115.0610
  8. Development of an IGVM Integrated Navigation System for Vehicular Lane-Level Guidance Services vol.5, pp.3, 2013, https://doi.org/10.11003/jpnt.2016.5.3.119
  9. AVE: Autonomous Vehicular Edge Computing Framework with ACO-Based Scheduling vol.66, pp.12, 2017, https://doi.org/10.1109/tvt.2017.2714704
  10. Smartphone-Based Vehicle Telematics: A Ten-Year Anniversary vol.18, pp.10, 2013, https://doi.org/10.1109/tits.2017.2680468
  11. The impact of drowsiness on in-vehicle human-machine interaction with head-up and head-down displays vol.77, pp.21, 2013, https://doi.org/10.1007/s11042-018-5966-9
  12. Research topics and implementation trends on automotive head-up display systems vol.12, pp.1, 2018, https://doi.org/10.1007/s12008-016-0350-3
  13. Perceived Importance of Automotive HUD Information Items: a Study With Experienced HUD Users vol.6, pp.None, 2013, https://doi.org/10.1109/access.2018.2828615
  14. A Study on User Experience of Automotive HUD Systems: Contexts of Information Use and User-Perceived Design Improvement Points vol.35, pp.20, 2013, https://doi.org/10.1080/10447318.2019.1587857
  15. A Survey on Vehicular Edge Computing: Architecture, Applications, Technical Issues, and Future Directions vol.2019, pp.None, 2013, https://doi.org/10.1155/2019/3159762
  16. Follow the Leader : Examining Real and Augmented Reality Lead Vehicles as Driving Navigational Aids vol.11, pp.2, 2013, https://doi.org/10.4018/ijmhci.2019040102
  17. Using Technologically Related Products From Other Domains as Inspirations for Technology-Push Product Concept Generation vol.143, pp.1, 2013, https://doi.org/10.1115/1.4047434
  18. Effects of full windshield head-up display on visual attention allocation vol.64, pp.10, 2013, https://doi.org/10.1080/00140139.2021.1912398
  19. Visual Enhancements for the Driver’s Information Search on Automotive Head-up Display vol.37, pp.18, 2021, https://doi.org/10.1080/10447318.2021.1908667