Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.5.738

Brain Correlates of Emotion for XR Auditory Content  

Park, Sangin (Industry-Academy Cooperation Team, Hanyang University)
Kim, Jonghwa (Department of Artificial Intelligence Research, Hansol Inticube)
Park, Soon Yong (Department of Smart City Research, Seoul Institute of Technology)
Mun, Sungchul (Department of Industrial Engineering, Jeonju University)
Publication Information
Journal of Broadcast Engineering / v.27, no.5, 2022 , pp. 738-750 More about this Journal
Abstract
In this study, we reviewed and discussed whether auditory stimuli with short length can evoke emotion-related neurological responses. The findings implicate that if personalized sound tracks are provided to XR users based on machine learning or probability network models, user experiences in XR environment can be enhanced. We also investigated that the arousal-relaxed factor evoked by short auditory sound can make distinct patterns in functional connectivity characterized from background EEG signals. We found that coherence in the right hemisphere increases in sound-evoked arousal state, and vice versa in relaxed state. Our findings can be practically utilized in developing XR sound bio-feedback system which can provide preference sound to users for highly immersive XR experiences.
Keywords
auditory stimulus; XR interface; brain connectivity; human factors; emotion;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Apple Glasses: here's everything we know so far, TechRadar News, https://www.techradar.com/news/apple-glasses(accessed Jul. 03, 2022).
2 Y. Xing et al., "Web XR User Interface Research: Design 3D Layout Framework in Static Websites," Applied Sciences, Vol.12, No.11, pp. 5600, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.3390/app12115600   DOI
3 H. Huang and C.-F. Lee, "Factors affecting usability of 3D model learning in a virtual reality environment," Interactive Learning Environments, Vol.30, No.5, pp. 848-861, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.1080/10494820.2019.1691605   DOI
4 S. Mun, M.-C. Park, and S. Yano, "Performance Comparison of a SSVEP BCI Task by Individual Stereoscopic 3d Susceptibility," International Journal of Human-Computer Interaction," Vol.29, No.12, pp.789-797, 2013, (accessed Jul. 8, 2022). doi: https://doi.org/10.1080/10447318.2013.765289   DOI
5 OpenBCI and Varjo Partner to Bring Neurotechnology to Spatial Computing, https://varjo.com/company-news/openbci-and-varjo-partnerto-bring-neurotechnology-to-spatial-computing (accessed Jul. 7, 2022).
6 S. Mun, E. S. Kim, and M. C. Park, "Effect of Mental Fatigue Caused by Mobile 3d Viewing on Selective Attention: An Erp Study," International Journal of Psychophysiology, Vol.94, No.3, pp.373-381, 2014, (accessed Jul. 7, 2022). doi: https://doi.org/10.1016/j.ijpsycho.2014.08.1389   DOI
7 E. Fox, "Perspectives from affective science on understanding the nature of emotion," Brain and Neuroscience Advances, Vol.2, pp. 1-8, 2018, (accessed Sep. 1, 2022). doi: https://doi.org/10.1177/2398212818812628   DOI
8 V. Putkinen et al., "Decoding Music-Evoked Emotions in the Auditory and Motor Cortex," Cerebral Cortex, Vol.31, No.5, pp.2549-2560, 2020, (accessed Jul. 7, 2022). doi: https://doi.org/10.1093/cercor/bhaa373   DOI
9 Apple Quietly Released One of The Most Impressive AR Room-mapping Tools, Road to VR - Virtual Reality News, https://www.roadtovr.com/apple-roomplan-ar-mapping (accessed Jul. 03, 2022).
10 DeepMotion Animate 3D, https://www.deepmotion.com (accessed Jul. 7, 2022).
11 S. Koelsch et al., "Neocortical Substrates of Feelings Evoked with Music in the ACC, Insula, and Somatosensory Cortex," Scientific Reports, Vol.11, No. 1, pp. 10119, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1038/s41598-021-89405-y   DOI
12 S. Sangnark et al., "Revealing Preference in Popular Music Through Familiarity and Brain Response," IEEE Sensors Journal, Vol.21, No. 13, pp. 14931-14940, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1109/JSEN.2021.3073040   DOI
13 S. Park and M. Whang, "Infrared camera-based non-contact measurement of brain activity from pupillary rhythms," Frontiers in Physiology, Vol.9, pp. 1400, 2018, (accessed Sep. 1, 2022). doi: https://doi.org/10.3389/fphys.2018.01400   DOI
14 S. Mun and M.-C. Park, "Affective Three-Dimensional Brain-Computer Interface Created Using a Prism Array-Based Display," Optical Engineering, Vol.53, No.12, pp.123105, 2014, (accessed Jul. 8, 2022). doi: https://doi.org/10.1117/1.OE.53.12.123105   DOI
15 G. Marion, G. M. D. Liberto, and S. A. Shamma, "The Music of Silence: Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics," The Journal of Neuroscience, Vol.41, No. 35, pp. 7435-7448, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1523/JNEUROSCI.0183-21.2021   DOI
16 S. Park, L. Kim, J. Ha, and S. Mun, "Infrared Webcam-Based Non-Contact Measurement of Event-Related Potentials from Event-Related Pupillary Responses: An Approach Focused on Mental Workload," Journal of Computational Design and Engineering, Vol.9, No. 4, pp.1327-1347, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.1093/jcde/qwac059   DOI
17 S. Mun, M.-C. Park, and S. Yano, "Evaluation of Viewing Experiences Induced by a Curved Three-Dimensional Display," Optical Engineering, Vol.54, No.10, pp.103104, 2015, (accessed Jul. 8, 2022). doi: https://doi.org/10.1117/1.OE.54.10.103104   DOI
18 S. Mun, et al., "Effects of Mental Workload on Involuntary Attention: A Somatosensory ERP Study," Neuropsychologia, Vol.106, pp.7-20, 2017, (accessed Jul. 8, 2022). doi: https://doi.org/10.1016/j.neuropsychologia.2017.08.021   DOI
19 L. Taruffi et al., "Effects of Sad and Happy Music on Mind-Wandering and the Default Mode Network," Scientific Reports, Vol.7, pp. 14396, 2017, (accessed Jul. 8, 2022). doi: https://doi.org/10.1038/s41598-017-14849-0   DOI
20 B. Lee and W. Son, "Evaluation of Human Factors in Extended Reality Content," Information and Communications Magazine, The Korean Institute of Commucations and Information Sciences, Sempember2021 (accessed Jul. 8, 2022).
21 S. Mun, E.-S. Kim, and M.-C. Park, "Effect of mental fatigue caused by mobile 3D viewing on selective attention: an ERP study," International Journal of Psychophysiology," Vol.94, No.3, pp.373-381, 2014, (accessed Jul. 8, 2022). doi: https://doi.org/10.1016/j.ijpsycho.2014.08.1389   DOI
22 H. Engelbregt et al., "Effects of binaural and monaural beat stimulation on attention and EEG," Experimental Brain Research, Vol.239, pp. 2781-2791, 2021, (accessed Sep. 1, 2022). doi: https://doi.org/10.1007/s00221-021-06155-z   DOI
23 L. A. Schmidt and L. J., "Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions," Cognition & Emotion, Vol.15, No. 4, pp. 487-500, 2001, (accessed Sep. 1, 2022). doi: https://doi.org/10.1080/02699930126048   DOI
24 J. H. Kim et al., "A research on EEG coherence variation by relaxation," Korean Journal of the science of Emotion & sensibility, Vol.13, No. 1, pp. 121-128, 2010 (accessed Jul. 8, 2022).
25 L. Taruffi, S. Skouras, C. Pehrs, and S. Koelsch, "Trait Empathy Shpaes Neural Responses Toward Sad Music," Cognitive, Affective, & Behavioral Neuroscience, Vol.21, No.1, pp.231-241, 2021, (accessed Jul. 8, 2022). doi: http://doi.org/10.3758/s13415-020-00861-x   DOI
26 S. Mun, et al., "SSVEP and ERP Measurement of Cognitive Fatigue Caused by Stereoscopic 3d," Neuroscience Letters, Vol.525, No.2, pp.89-94, 2012, (accessed Jul. 8, 2022). doi: https://doi.org/10.1016/j.neulet.2012.07.049   DOI