Educational Technology International

The Korean Society for Education Technology (한국교육공학회)
권호 표지
DOI QR코드

DOI QR Code

Prototyping Training Program in Immersive Virtual Learning Environment with Head Mounted Displays and Touchless Interfaces for Hearing-Impaired Learners

  • Received : 2017.03.27
  • Accepted : 2017.04.05
  • Published : 2017.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of the study was to identify key design features of virtual reality with head-mounted displays (HMD) and touchless interface for the hearing-impaired and hard-of-hearing learners. The virtual reality based training program was aimed to help hearing-impaired learners in machine operating learning, which requires spatial understanding to operate. We developed an immersive virtual learning environment prototype with an HMD (Oculus Rift) and a touchless natural user interface (Leap Motion) to identify the key design features required to enhance virtual reality for the hearing-impaired and hard-of-hearing learners. Two usability tests of the prototype were conducted, which revealed that several features in the system need revision and that the technology presents an enormous potential to help hearing-impaired learners by providing realistic and immersive learning experiences. After the usability tests of hearing-impaired students' exploring the 3D virtual space, interviews were conducted, which also established that further revision of the system is needed, which would take into account the learners' physical as well as cognitive characteristics.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government(NRF-2014S1A5A2A03064906).

References

  1. Adams Becker, S., Freeman, A., Giesinger Hall, C., Cummins, M., and Yuhnke, B. (2016). NMC/CoSN Horizon Report: 2016 K-12 Edition. Austin, Texas: The New Media Consortium.
  2. Amiel, T., & Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educational Technology & Society, 11(4), 29-40.
  3. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645.
  4. Blatto-Vallee, G., Kelly, R. R., Gaustad, M. G., Porter, J., & Fonzi, J. (2007). Visual-Spatial Representation in Mathematical Problem Solving by Deaf and Hearing Students. Journal of Deaf Studies and Deaf Education, 12(4), 432-448.
  5. Coelho, J. C., & Verbeek, F. J. (2014). Pointing task evaluation of leap motion controller in 3d virtual environment. Creating the Difference, 78, 78-85.
  6. Connors, E. C., Yazzolino, L. A., Sanchez, J., & Merabet, L. B. (2013). Development of an audio-based virtual gaming environment to assist with navigation skills in the blind. JoVE (Journal of Visualized Experiments)(73).
  7. Eden, S. (2008). The effect of 3D virtual reality on sequential time perception among deaf and hard-of-hearing children. European Journal of Special Needs Education, 23(4), 349-363.
  8. Eden, S., & Ingber, S. (2014). Virtual Environments as a Tool for Improving Sequence Ability of Deaf and Hard of Hearing Children. American Annals of the Deaf, 159(3), 284-295. doi:10.1353/aad.2014.0025
  9. Ehrlich, S. B., Levine, S. C., & Goldin-Meadow, S. (2006). The importance of gesture in children's spatial reasoning. Developmental Psychology, 42(6), 1259.
  10. Emmorey, K., Kosslyn, S. M., & Bellugi, U. (1993). Visual imagery and visual-spatial language: Enhanced imagery abilities in deaf and hearing ASL signers. Cognition, 46(2), 139-181.
  11. Han, I., & Black, J. B. (2011). Incorporating haptic feedback in simulation for learning physics. Computers & Education, 57(4), 2281-2290.
  12. Hoberman, P., Krum, D. M., Suma, E. A., & Bolas, M. (2012). Immersive training games for smartphone-based head mounted displays. Paper presented at the Virtual Reality Short Papers and Posters (VRW), 2012 IEEE.
  13. Ingber, S., & Eden, S. (2011). Enhancing Sequential Time Perception and Storytelling Ability of Deaf and Hard of Hearing Children. American Annals of the Deaf, 156(4), 391-401. doi:10.1353/aad.2011.0033
  14. Klima, E. S., Tzeng, O. J. L., Fok, Y. Y. A., Bellugi, U., Corina, D., & Bettger, J. G. (1999). From sign to script: Effects of linguistic experience on perceptual categorization. Journal of Chinese Linguistics Monograph Series(13), 96-129.
  15. Lahav, O., Schloerb, D., Kumar, S., & Srinivasan, M. (2012). A virtual environment for people who are blind-a usability study. Journal of assistive technologies, 6(1), 38-52.
  16. Merhi, O., Faugloire, E., Flanagan, M., & Stoffregen, T. A. (2007). Motion sickness, console video games, and head-mounted displays. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(5), 920-934.
  17. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of educational psychology, 91(2), 358.
  18. Nabiyouni, M., Laha, B., & Bowman, D. (2014). Poster: Designing effective travel techniques with bare-hand interaction. Paper presented at the 3D User Interfaces (3DUI), 2014 IEEE Symposium on.
  19. Passig, D., & Eden, S. (2000). Enhancing the induction skill of deaf and hard-of-hearing children with virtual reality technology. Journal of Deaf Studies and Deaf Education, 5(3), 277-285.
  20. Passig, D., & Eden, S. (2001). Virtual reality as a tool for improving spatial rotation among deaf and hard-of-hearing children. CyberPsychology & Behavior, 4(6), 681-686.
  21. Powers, D. A., & Darrow, M. (1994). Special education and virtual reality: Challenges and possibilities. Journal of Research on Computing in Education, 27(1), 111.
  22. Segal, A., Tversky, B., & Black, J. (2014). Conceptually congruent actions can promote thought. Journal of Applied Research in Memory and Cognition, 3(3), 124-130.
  23. Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators & Virtual Environments, 10(3), 266-281. doi:10.1162/105474601300343603
  24. Sharples, S., Cobb, S., Moody, A., & Wilson, J. R. (2008). Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays, 29(2), 58-69. doi:http://dx.doi.org/10.1016/j.displa.2007.09.005
  25. Smedley, T. M., & Higgins, K. (2005). Virtual technology: Bringing the world into the special education classroom. Intervention in School and Clinic, 41(2), 114-119.
  26. Smith, L., & Gasser, M. (2005). The development of embodied cognition: Six lessons from babies. Artificial life, 11(1-2), 13-29.
  27. Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments, 7(3), 225-240.
  28. Wood, J., Magennis, M., Arias, E. F. C., Gutierrez, T., Graupp, H., & Bergamasco, M. (2003). The design and evaluation of a computer game for the blind in the GRAB haptic audio virtual environment. Proceedings of Eurohpatics.
  29. Xu, X., Chen, K. B., Lin, J.-H., & Radwin, R. G. (2015). The accuracy of the Oculus Rift virtual reality head-mounted display during cervical spine mobility measurement. Journal of Biomechanics, 48(4), 721-724. doi:http://dx.doi.org/10.1016/j.jbiomech.2015.01.005