Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

A Study on Manipulating Method of 3D Game in HMD Environment by using Eye Tracking

HMD(Head Mounted Display)에서 시선 추적을 통한 3차원 게임 조작 방법 연구

  • Published : 2008.03.25
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, many researches about making more comfortable input device based on gaze detection technology have been done in human computer interface. However, the system cost becomes high due to the complicated hardware and there is difficulty to use the gaze detection system due to the complicated user calibration procedure. In this paper, we propose a new gaze detection method based on the 2D analysis and a simple user calibration. Our method used a small USB (Universal Serial Bus) camera attached on a HMD (Head-Mounted Display), hot-mirror and IR (Infra-Red) light illuminator. Because the HMD is moved according to user's facial movement, we can implement the gaze detection system of which performance is not affected by facial movement. In addition, we apply our gaze detection system to 3D first person shooting game. From that, the gaze direction of game character is controlled by our gaze detection method and it can target the enemy character and shoot, which can increase the immersion and interest of game. Experimental results showed that the game and gaze detection system could be operated at real-time speed in one desktop computer and we could obtain the gaze detection accuracy of 0.88 degrees. In addition, we could know our gaze detection technology could replace the conventional mouse in the 3D first person shooting game.

최근에 휴먼 컴퓨터 인터페이스 분야에서 사용자의 시선 위치를 파악하여 더욱 편리한 입력장치를 구축하고자 하는 연구가 많이 진행되고 있다. 하지만 복잡한 하드웨어 구성으로 제품의 가격이 매우 비싸고, 까다로운 사용자 캘리브레이션 과정으로 인해 시스템의 사용에 어려움을 겪는다. 본 논문에서는 HMD(Head Mounted Display)에 USB 카메라와 적외선을 반사시키는 hot-mirror와 적외선 조명을 이용한 시선 추적 모듈을 부착하고, 이를 통해 획득한 눈 영상의 2차원적인 분석과 간단한 사용자 캘리브레이션 과정을 통해 시선 위치를 파악하는 방법을 제안한다. HMD는 사용자의 얼굴 움직임과 함께 움직이므로, 얼굴움직임에 영향을 받지 않는 시선 추적 시스템을 구현할 수 있다. 또한, 시선 추적 시스템을 3차원 1인칭 슈팅 게임에 적응하여, 캐릭터의 시선 방향을 조정하고, 적 캐릭터를 조준하여 사격이 가능하도록 하여, 게임의 몰입감과 흥미성을 높일 수 있게 하였다. 실험 결과, 한 대의 데스크톱 컴퓨터 환경에서 게임과 시선 추적 시스템이 실시간으로 동작 가능했으며, 약 $0.88^{\circ}$의 시선 위치 추출 오차를 보였다. 또한 3차원 1인칭 슈팅게임에서 일반 마우스의 역할을 시선 추적 시스템이 문제없이 대신할 수 있음을 확인하였다.

Keywords

References

  1. Jacob, R. J. K, 'Eye Movement-Based Human-Computer Interaction Techniques : Toward Non-Command Interfaces', Advances in Human-Computer Interaction, H. R. Hartson and D. Hix, Editors., Ablex Publishing Co., pp. 151-190, 1993
  2. L. E. Sibert, R. J. K. Jacob, 'Evaluation of Eye Gaze Interaction', Proc. of the CHI, ACM in New York, pp281-288, 2000
  3. Ohno, T. 'Quick Menu Selection Task with Eye Mark', Transactions of Information Processing Society of Japan, vol. 40, no. 2 , pp.602-612, 1999
  4. Yamato, M., Monden, A., Matsumoto, K., Inoue, K. and Torii, K, 'Quick Button Selection with Eye Gazing for General GUI Environments', International Conference on Software: Theory and Practice, August 2000
  5. Andrew T. Duchowski, Vinay Shiva shankariah, Tim Rawls, Anand K.Gramopadhye, Brian J. Melloy, Barbara Kanki, 'Binocular Eye Tracking in Virtual Reality for Inspection Training', ACM Press New York, NY, USA 2002
  6. Jeong Jun Lee, Kang Ryoung Park, Jaihie Kim, 'Gaze detection system under HMD environment for user interface', the Joint Conference of ICANN/ICONIP, June 2003
  7. K.Ando et al., 'Effects of phencyclidine, secobarbital and diazepam on eye tracking in rhesus monkeys', Psychopharmacology, Springer
  8. Daniel Cheng and Roel Vertegaal, 'An Eye for an Eye: A Performance Evaluation Comparison of the LC Technologies and Tobii Eye Trackers', the Association for Computing Machinery, Inc., pp61, 2004
  9. Shen-Wen Shih, Jin Liu, 'A Novel Approach to 3-D Gaze Tracking Using Stereo Cameras', IEEE Transactions on system, man, and Cybernetics Part B: Cybernetics, VOL, 34, NO.1, February 2004
  10. Dong Hyun Yoo and Myung Jin Chung, 'Non-Intrusive Eye Gaze Estimation without Knowledge of Eye Pose', Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004
  11. 이의철, 박강령, '눈동자 시선 추적에 의한 3차원 1인칭 슈팅 게임', 한국정보처리학회 논문지, 제12-B권 제4호, pp. 465-472, 2005년 8월 https://doi.org/10.3745/KIPSTB.2005.12B.4.465
  12. http://www.serrata.com.au/iglassesvga.html#i-glasses%20SVGA%203D, accessed on 2008.2.25
  13. http://www.logitech.com, accessed on 2008.2.25
  14. http://www.kodak.com/global/en/professional/support/techPubs/b3akic/b3akic.jhtml, accessed on 2008.2.25
  15. John G. Daugman, 'High confidence visual recognition of personals by a test of statistical independence,' IEEE Trans. Pattern Anal. Machine Intell., Vol. 15, No. 11, pp.1148-1160, 1993 https://doi.org/10.1109/34.244676
  16. Gregory A. Baxes, 'Digital Image Processing-Principles and Application', Wiley
  17. Edward Angel, 'Interactive Computer Graphics', A Top-Down Approach with OpenGL, 2nd Edition
  18. 박현애, 박강령, '휴대폰에서의 홍채인식을 위한 고속 홍채검출에 관한 연구', 대한전자공학회 논문지, 제 43권 SP편 제 2 호, pp. 19-29, 2006년 3월
  19. N. Otsu. A threshold selection method from gray-level histograms. IEEE Trans. on System, Man and Cybernetics, 9(1):62--66, 1979 https://doi.org/10.1109/TSMC.1979.4310076
  20. Rafael C.Gonzalez, Richard E. Woods, 'Digital Image Processing Second Edition', 2002 by Prentice-Hall, Inc
  21. Richard N. Aslin and Bob McMurray, 'Automated Corneal-Reflection Eye Tracking in Infancy: Methodological Developments and Applications to Cognition', Lawrence Erlbaum Associates, Inc., INFANCY, 6(2), 155-163, 2004
  22. John David Smith, 'ViewPointer: Lightweight Calibration-Free Eye Tracking for Ubiquitous Handsfree Deixis', A thesis of Master of Science, Queen's University, September 2005
  23. Sheng-Wen Shih, Yu-Te Wu, Jin Liu, 'A Calibration-Free Gaze Tracking Technique', 15th International Conference on Pattern Recognition (ICPR'00) - Volume 4 p. 4201
  24. Jukka Hakkinen, Tero Vuori, Monika Puhakka, 'Postural Stability and Sickness Symptoms after HMD Use', IEEE International Conference on Systems, Man and Cybernetics, pp. 4-7, 2002
  25. Clark, M.R, Stark, L., 'Time optimal behavior of human saccadic eye movement.,' IEEE Trans. on Automatic Control AC-20 (1975), 345-348
  26. Eidos, 'Athens 2004 PC Games', Oct 14, 2004
  27. 'Method and Apparatus for Tracking Gaze Position', 미국 특허 출원중 (출원 번호 : 11/91813, 출원일 : 2007.12.06), 발명자(이의철 외)
  28. Eui Chul Lee, Kang Ryoung Park, 'A Robust Eye Gaze Tracking Method based on Virtual Eyeball Model', Machine Vision and Applications, accepted for publication
  29. Eui Chul Lee, Kang Ryoung Park, 'A Study on Eye Gaze Estimation Method Based on Cornea Model of Human Eye', Lecture Notes in Computer Science (MIRAGE 2007), Vol. 4418, pp. 307-317, INRIA Rocquencourt, France, March, 28 - 30, 2007 https://doi.org/10.1007/978-3-540-71457-6_28
  30. You Jin Ko, Eui Chul Lee, Kang Ryoung Park, 'A Robust Gaze Detection Method by Compensating for Facial Movements Based on Corneal Specularities', Pattern Recognition Letters, accepted for publication
  31. You Jin Ko, Eui Chul Lee, Kang Ryoung Park, 'A study on Robust eye gaze tracking system allowing the natural facial movements', ACCV'07 Workshop on Multi-dimensional and Multi-view Image Processing, Tokyo, 19 Nov. 2007
  32. 고유진, 이의철, 박강령, '적외선 환경에서 얼굴 및 눈동자 움직임이 자유로운 시선 위치 추적에 대한 연구', 2006년 한국 차세대컴퓨팅학회 논문 발표회, 일산 KINTEX, 2006.11.16-17
  33. 이의철, 박강령, '사용자 시선 추적을 통한 3차원 FPS 게임 화면 조정', 2006년 한국 차세대컴퓨팅학회 논문 발표회, 일산 KINTEX, 2006.11.16-17
  34. Eui Chul Lee, Kang Ryoung Park, Min Cheol Whang, Junseok Park, 'Robust Gaze Tracking Method for Stereoscopic Virtual Reality System', Lecture Notes in Computer Science (HCI International 2007), Vol. 4552, pp. 700-709, Beijing, China, July 22-27, 2007 https://doi.org/10.1007/978-3-540-73110-8_76
  35. Eui Chul Lee, Kang Ryoung Park, Min Cheol Whang, Joa Sang Lim, 'Vision-based Gaze Estimation Method For Stereoscopic Display Systems', 10th International Federation of Automatic Control (IFAC), Ritz-Carlton Hotel, Seoul, Korea, 4-6 Sep., 2007