Browse > Article
http://dx.doi.org/10.15701/kcgs.2019.25.3.11

Design of Immersive Walking Interaction Using Deep Learning for Virtual Reality Experience Environment of Visually Impaired People  

Oh, Jiseok (Department of Software, Catholic University of Pusan)
Bong, Changyun (Department of Software, Catholic University of Pusan)
Kim, Jinmo (Division of Computer Engineering, Hansung University)
Publication Information
Journal of the Korea Computer Graphics Society / v.25, no.3, 2019 , pp. 11-20 More about this Journal
Abstract
In this study, a novel virtual reality (VR) experience environment is proposed for enabling walking adaptation of visually impaired people. The core of proposed VR environment is based on immersive walking interactions and deep learning based braille blocks recognition. To provide a realistic walking experience from the perspective of visually impaired people, a tracker-based walking process is designed for determining the walking state by detecting marching in place, and a controller-based VR white cane is developed that serves as the walking assistance tool for visually impaired people. Additionally, a learning model is developed for conducting comprehensive decision-making by recognizing and responding to braille blocks situated on roads that are followed during the course of directions provided by the VR white cane. Based on the same, a VR application comprising an outdoor urban environment is designed for analyzing the VR walking environment experience. An experimental survey and performance analysis were also conducted for the participants. Obtained results corroborate that the proposed VR walking environment provides a presence of high-level walking experience from the perspective of visually impaired people. Furthermore, the results verify that the proposed learning algorithm and process can recognize braille blocks situated on sidewalks and roadways with high accuracy.
Keywords
immersive virtual reality; walking interaction; deep learning; visually impaired people;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 H. Joo, T. Simon, and Y. Sheikh, "Total capture: A 3d deformation model for tracking faces, hands, and bodies," in Proceedings of The 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ser. CVPR '18, vol. abs/1801.01615. Washington, DC, USA: IEEE Computer Society, 18-22 June 2018, pp. 8320-8329. [Online]. Available: http://arxiv.org/abs/1801.01615
2 S. Marwecki, M. Brehm, L. Wagner, L.-P. Cheng, F. F. Mueller, and P. Baudisch, "Virtualspace - overloading physical space with multiple virtual reality users," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ser. CHI '18. New York, NY, USA: ACM, 21-26 April 2018, pp. 241:1-241:10. [Online]. Available: http://doi.acm.org/10.1145/3173574.3173815
3 S. Park, W. Park, H. Heo, and J. Kim, "A study on presence of collaboration based multi-user interaction in immersive virtual reality," Journal of the Korea Computer Graphics Society, vol. 24, no. 3, pp. 11-20, 2018. [Online]. Available: https://doi.org/10.15701/kcgs.2018.24.3.11   DOI
4 E. Remelli, A. Tkach, A. Tagliasacchi, and M. Pauly, "Low-dimensionality calibration through local anisotropic scaling for robust hand model personalization," 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2554-2562, 22-29 October 2017.
5 Z.-C. Dong, X.-M. Fu, C. Zhang, K. Wu, and L. Liu, "Smooth assembled mappings for large-scale real walking," ACM Trans. Graph., vol. 36, no. 6, pp. 211:1-211:13, Nov. 2017. [Online]. Available: http://doi.acm.org/10.1145/3130800.3130893   DOI
6 J. Lee, K. Jeong, and J. Kim, "Mave: Maze-based immersive virtual environment for new presence and experience," Computer Animation and Virtual Worlds, vol. 28, no. 3-4, p. e1756, 2017.   DOI
7 H. Wu, Y. Wang, J. Qiu, J. Liu, and X. L. Zhang, "User-defined gesture interaction for immersive vr shopping applications," Behaviour & Information Technology, vol. 0, no. 0, pp. 1-16, 2018. [Online]. Available: https://doi.org/10.1080/0144929X.2018.1552313   DOI
8 G. Yu, H.-J. Kim, H.-S. Kim, and J. Lee, "Virtual home training - virtual reality small scale rehabilitation system," Journal of the Korea Computer Graphics Society, vol. 24, no. 3, pp. 93-100, 2018. [Online]. Available: https://doi.org/10.15701/kcgs.2018.24.3.93   DOI
9 C. Pacchierotti, F. Chinello, M. Malvezzi, L. Meli, and D. Prattichizzo, "Two finger grasping simulation with cutaneous and kinesthetic force feedback," in Haptics: Perception, Devices, Mobility, and Communication (EuroHaptics 2012), P. Isokoski and J. Springare, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 13-15 June 2012, pp. 373-382.
10 M. Slater and M. V. Sanchez-Vives, "Enhancing our lives with immersive virtual reality," Frontiers in Robotics and AI, vol. 3, p. 74, 2016.
11 M. Slater, M. Usoh, and A. Steed, "Taking steps: The influence of a walking technique on presence in virtual reality," ACM Trans. Comput.-Hum. Interact., vol. 2, no. 3, pp. 201-219, Sept. 1995.   DOI
12 H.-S. Kim, H.-J. Kim, Y.-S. Lee, and J. Lee, "Criminal profiling simulation training and assessment system based on virtual," Journal of the Korea Computer Graphics Society, vol. 24, no. 3, pp. 83-92, 2018. [Online]. Available: https://doi.org/10.15701/kcgs.2018.24.3.83   DOI
13 M. Kim, C. Jeon, and J. Kim, "A study on immersion and presence of a portable hand haptic system for immersive virtual reality," Sensors, vol. 17, no. 5, p. 1141, 2017. [Online]. Available: http://www.mdpi.com/1424-8220/17/5/1141   DOI
14 M. Kim, J. Lee, C. Jeon, and J. Kim, "A study on interaction of gaze pointer-based user interface in mobile virtual reality environment," Symmetry, vol. 9, no. 9, p. 189, 2017. [Online]. Available: http://www.mdpi.com/2073-8994/9/9/189   DOI
15 D. Prattichizzo, F. Chinello, C. Pacchierotti, and M. Malvezzi, "Towards wearability in fingertip haptics: A 3-dof wearable device for cutaneous force feedback," IEEE Transactions on Haptics, vol. 6, no. 4, pp. 506-516, Oct 2013.   DOI
16 M. Slater and M. Usoh, "Simulating peripheral vision in immersive virtual environments," Computers & Graphics, vol. 17, no. 6, pp. 643 - 653, 1993.   DOI
17 V. Vinayagamoorthy, M. Garau, A. Steed, and M. Slater, "An eye gaze model for dyadic interaction in an immersive virtual environment: Practice and experience," Computer Graphics Forum, vol. 23, no. 1, pp. 1-11, 2004.   DOI
18 C. Carvalheiro, R. Nobrega, H. da Silva, and R. Rodrigues, "User redirection and direct haptics in virtual environments," in Proceedings of the 2016 ACM on Multimedia Conference, ser. MM '16. New York, NY, USA: ACM, 15-19 October 2016, pp. 1146-1155.
19 S. Han and J. Kim, "A study on immersion of hand interaction for mobile platform virtual reality contents," Symmetry, vol. 9, no. 2, p. 22, 2017.   DOI
20 W. Zhao, J. Chai, and Y.-Q. Xu, "Combining markerbased mocap and rgb-d camera for acquiring high-fidelity hand motion data," in Proceedings of the ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, ser. SCA '12. Aire-la-Ville, Switzerland, Switzerland: Eurographics Association, 29-31 July 2012, pp. 33-42.
21 A. Jayasiri, S. Ma, Y. Qian, K. Akahane, and M. Sato, "Desktop versions of the string-based haptic interface-spidar," in 2015 IEEE Virtual Reality (VR). IEEE, 23-27 March 2015, pp. 199-200.
22 D. Leonardis, M. Solazzi, I. Bortone, and A. Frisoli, "A 3-rsr haptic wearable device for rendering fingertip contact forces," IEEE Transactions on Haptics, vol. 10, no. 3, pp. 305-316, July 2017.   DOI
23 B. G. Witmer, C. J. Jerome, and M. J. Singer, "The factor structure of the presence questionnaire," Presence: Teleoper. Virtual Environ., vol. 14, no. 3, pp. 298-312, jun 2005.   DOI
24 K. Jeong, S. Han, D. Lee, and J. Kim, "A Study on Virtual Reality Techniques for Immersive Traditional Fairy Tale Contents Production," Journal of the Korea Computer Graphics Society, vol. 22, pp. 43-52, 2016.   DOI
25 J. Lee, M. Kim, and J. Kim, "A study on immersion and vr sickness in walking interaction for immersive virtual reality applications," Symmetry, vol. 9, no. 5, p. 78, 2017.   DOI
26 C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," CoRR, vol. abs/1512.00567, 2015. [Online]. Available: http://arxiv.org/abs/1512.00567