Browse > Article
http://dx.doi.org/10.15701/kcgs.2019.25.1.23

Motion Sickness Measurement and Analysis in Virtual Reality using Deep Neural Networks Algorithm  

Jeong, Daekyo (Sejong University)
Yoo, Sangbong (Sejong University)
Jang, Yun (Sejong University)
Publication Information
Journal of the Korea Computer Graphics Society / v.25, no.1, 2019 , pp. 23-32 More about this Journal
Abstract
Cybersickness is a symptom of dizziness that occurs while experiencing Virtual Reality (VR) technology and it is presumed to occur mainly by crosstalk between the sensory and cognitive systems. However, since the sensory and cognitive systems cannot be measured objectively, it is difficult to measure cybersickness. Therefore, methodologies for measuring cybersickness have been studied in various ways. Traditional studies have collected answers to questionnaires or analyzed EEG data using machine learning algorithms. However, the system relying on the questionnaires lacks objectivity, and it is difficult to obtain highly accurate measurements with the machine learning algorithms. In this work, we apply Deep Neural Network (DNN) deep learning algorithm for objective cybersickness measurement from EEG data. We also propose a data preprocessing for learning and network structures allowing us to achieve high performance when learning EEG data with the deep learning algorithms. Our approach provides cybersickness measurement with an accuracy up to 98.88%. Besides, we analyze video characteristics where cybersickness occurs by examining the video segments causing cybersickness in the experiments. We discover that cybersickness happens even in unusually persistent changes in the darkness such as the light in a room keeps switching on and off.
Keywords
Virtual Reality (VR); Cybersickness; EEG; Deep Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. J. LaViola Jr. A discussion of cybersickness in virtual environments. ACM SIGCHI Bulletin, 32(1):47-56, 2000.   DOI
2 K. Nesbitt, S. Davis, K. Blackmore, and E. Nalivaiko. Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays, 48:1-8, 2017.   DOI
3 B. Keshavarz and H. Hecht. Pleasant music as a countermeasure against visually induced motion sickness. Applied ergonomics, 45(3):521-527, 2014.   DOI
4 Y.-H. Yu, P.-C. Lai, L.-W. Ko, C.-H. Chuang, B.-C. Kuo, and C.-T. Lin. An eeg-based classification system of passenger's motion sickness level by using feature extraction/selection technologies. In Neural Networks (IJCNN), The 2010 International Joint Conference on, pp. 1-6. IEEE, 2010.
5 C.-S. Wei, L.-W. Ko, S.-W. Chuang, T.-P. Jung, and C.-T. Lin. Eegbased evaluation system for motion sickness estimation. In Neural Engineering (NER), 2011 5th International IEEE/EMBS Conference on, pp. 100-103. IEEE, 2011.
6 C.-T. Lin, S.-F. Tsai, L.-W. Ko, et al. Eeg-based learning system for online motion sickness level estimation in a dynamic vehicle environment. IEEE transactions on neural networks and learning systems, 24(10):1689-1700, 2013.   DOI
7 K. G. Hartmann, R. T. Schirrmeister, and T. Ball. Hierarchical internal representation of spectral features in deep convolutional networks trained for eeg decoding. In Brain-Computer Interface (BCI), 2018 6th International Conference on, pp. 1-6. IEEE, 2018.
8 R. Schirrmeister, L. Gemein, K. Eggensperger, F. Hutter, and T. Ball. Deep learning with convolutional neural networks for decoding and visualization of eeg pathology. In Signal Processing in Medicine and Biology Symposium (SPMB), 2017 IEEE, pp. 1-7. IEEE, 2017.
9 O. Dressler, G. Schneider, G. Stockmanns, and E. Kochs. Awareness and the eeg power spectrum: analysis of frequencies. British journal of anaesthesia, 93(6):806-809, 2004.   DOI
10 B. Hjorth. Eeg analysis based on time domain properties. Electroencephalography and clinical neurophysiology, 29(3):306-310, 1970.   DOI
11 T.-P. Jung, S. Makeig, M. Stensmo, and T. J. Sejnowski. Estimating alertness from the eeg power spectrum. IEEE transactions on biomedical engineering, 44(1):60-69, 1997.   DOI
12 S. Liang, C. Lin, R. Wu, Y. Chen, T. Huang, and T. Jung. Monitoring driver's alertness based on the driving performance estimation and the eeg power spectrum analysis. In Conf Proc IEEE Eng Med Biol Soc, vol. 6, pp. 5738-5741, 2005.
13 A. J. Bell and T. J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural computation, 7(6):1129-1159, 1995.   DOI
14 P. Comon. Independent component analysis, a new concept? Signal processing, 36(3):287-314, 1994.   DOI
15 C. Jutten and J. Herault. Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture. Signal processing, 24(1):1-10, 1991.   DOI
16 J. A. Uriguen and B. Garcia-Zapirain. Eeg artifact removal-state-ofthe- art and guidelines. Journal of neural engineering, 12(3):031001, 2015.   DOI
17 B.-C. Kuo and K.-Y. Chang. Feature extractions for small sample size classification problem. IEEE Transactions on Geoscience and Remote Sensing, 45(3):756-764, 2007.   DOI
18 T.-W. Lee, M. Girolami, and T. J. Sejnowski. Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural computation, 11(2):417-441, 1999.   DOI
19 S. Makeig, T.-P. Jung, A. J. Bell, D. Ghahremani, and T. J. Sejnowski. Blind separation of auditory event-related brain responses into independent components. Proceedings of the National Academy of Sciences, 94(20):10979-10984, 1997.   DOI
20 M. Othman, A. Wahab, I. Karim, M. A. Dzulkifli, and I. F. T. Alshaikli. Eeg emotion recognition based on the dimensional models of emotions. Procedia-Social and Behavioral Sciences, 97:30-37, 2013.   DOI
21 E. Durall, T. Leinonen, B. Gros, and T. Rodriguez-Kaarto. Reflection in learning through a self-monitoring device: Design research on eeg self-monitoring during a study session. Designs for Learning, 9(1), 2017.
22 D. O. Bos et al. Eeg-based emotion recognition. The Influence of Visual and Auditory Stimuli, 56(3):1-17, 2006.
23 P. C. Petrantonakis and L. J. Hadjileontiadis. Emotion recognition from eeg using higher order crossings. IEEE Transactions on Information Technology in Biomedicine, 14(2):186-197, 2010.   DOI
24 Y.-P. Lin, C.-H. Wang, T.-P. Jung, T.-L. Wu, S.-K. Jeng, J.-R. Duann, and J.-H. Chen. Eeg-based emotion recognition in music listening. IEEE Transactions on Biomedical Engineering, 57(7):1798-1806, 2010.   DOI
25 EMOTIV. Developers - emotiv. Updated May 28, 2018, https://www.emotiv.com/developer/, 2018.
26 Y. Liu, O. Sourina, and M. K. Nguyen. Real-time eeg-based human emotion recognition and visualization. In 2010 international conference on cyberworlds, pp. 262-269. IEEE, 2010.
27 D. Nie, X. Wang, L. Shi, and B. Lu. Eeg-based emotion recognition during watching movies. In 2011 5th International IEEE/EMBS Conference on Neural Engineering, pp. 667-670, April 2011. doi: 10. 1109/NER.2011.5910636   DOI
28 W. Zheng, B. Dong, and B. Lu. Multimodal emotion recognition using eeg and eye tracking data. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5040-5043, Aug 2014. doi: 10.1109/EMBC.2014.6944757   DOI
29 Supercell. Clash of clans: Hog rider 360. Updated November 19, 2016, https://www.youtube.com/watch?v=yVLfEHXQk08, 2016.
30 Google Spotlight Stories. 360 google spotlight stories: Pearl. Updated May 20, 2016, https://www.youtube.com/watch?v=WqCH4DNQBUA, 2016.
31 ApexTV. 2017. 360 Jeff The Killer VR Horror Experience. Updated October 28, 2017, https://www.youtube.com/watch?v=icV-OzwKS-k.
32 D. E. Rumelhart, J. L. McClelland, P. R. Group, et al. Parallel distributed processing, vol. 1. MIT press Cambridge, MA, 1987.
33 M. Minsky and S. A. Papert. Perceptrons: An introduction to computational geometry. MIT press, 2017.
34 FOX International. The walking dead - vr 360 video. Updated October 20, 2016, https://www.youtube.com/watch?v=nRQsnqd2Vs, 2016.