Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Classifying and Representing Emotions based on EEG Data Collected with Wearable Devices

웨어러블 기구로 측정된 EEG 데이터 기반 정서 분류 및 표상

  • Received : 2024.09.02
  • Accepted : 2024.11.10
  • Published : 2024.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This research investigated the classification and representation of discrete emotions based on electroencephalogram (EEG) data collected using wearable devices in response to emotional videos. To achieve this objective, a previously published dataset was reanalyzed using multivariate analyses, which are suitable for handling high-dimensional data features, minimizing information loss, and extracting core affect dimensions. First, the cross-participant classification was conducted to confirm the consistency of the emotional responses across the participants. The classification results were evaluated using a confusion matrix, indicating the correct and incorrect classifications for each emotion video. Second, based on the core affect model, multidimensional scaling (MDS) was performed to examine the representation of emotions within a two-dimensional space. Finally, representational similarity analysis (RSA) was conducted to determine the most descriptive type of data-behavioral, physiological, or EEG-in characterizing emotional dimensions. The classification analysis revealed that emotional conditions were successfully classified across participants, suggesting that the emotion-eliciting EEG response is shared across individuals. The results of MDS indicated that the EEG data collected in response to emotional stimuli exhibited clear representation along the arousal dimension. The results of RSA demonstrated that behavioral data best described the valence dimension, while EEG and physiological data best explained the arousal dimension. This study offers evidence supporting the core affect theory through EEG data collected via wearable devices in response to video stimuli.

본 연구는 정서가 어떻게 표상되고 그 구조가 어떠한지 파악하기 위한 목적으로 진행되었다. 이에 정서 영상에서 유발된 정서에 대해 웨어러블 기구 기반 EEG 반응이 어떻게 분류되고 표상되는지 알아보고자 하였다. 이를 위해 참가자간 분류분석으로 참가자들 간의 정서 반응의 일관성을 확인하였다. 또한 분류분석의 결과로 도출된 정서 영상별 정확 및 오분류를 오차행렬을 통해 기술하였다. 다음으로, 핵심정서 모델을 기반으로 EEG 데이터에 다차원척도법을 적용하여 정서가 정서가와 각성가 두 가지 차원에서 어떻게 표상되는지 확인하였다. 마지막으로, 표상 유사성 분석을 통해 행동 데이터, 생리 측정 데이터, EEG 데이터 중 정서 차원을 가장 잘 설명하는 데이터가 무엇인지 확인하였다. 참가자간 분류분석 결과, 정서유형이 유의미하게 분류되었고 이는 EEG 데이터에 정서에 따라 참가자들 간 공유되는 패턴이 있음을 시사한다. 다차원척도법 결과, 정서 자극에 따른 EEG 데이터가 각성가 차원에서 표상이 잘 이루어졌다. 표상 유사성 분석 결과, 정서가 차원은 행동 데이터가, 각성가 차원은 EEG 데이터와 생리 측정 데이터가 잘 설명하였다. 본 연구는 웨어러블 기구를 통해 측정된 영상 자극에 따른 EEG 데이터를 사용해 기존의 정서 이론과 부합하는 결과를 얻은 것에 의의가 있다.

Keywords

Acknowledgement

이 논문은 한국연구재단 4단계 BK21 사업(전북대학교 심리학과)의 지원을 받아 수행된 연구임.

References

  1. Balconi, M., & Lucchiari, C. (2008). Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis. International Journal of Psychophysiology, 67(1), 41-46. DOI: 10.1016/j.ijpsycho.2007.10.002
  2. Baucom, L. B., Wedell, D. H., Wang, J., Blitzer, D. N., & Shinkareva, S. V. (2012). Decoding the neural representation of affective states. NeuroImage, 59(1), 718-727. DOI: 10.1016/j.neuroimage.2011.07.037
  3. Chae, S., & Kim, S. (2016). Development of a countinuous emotion recognition model using fractal dimensions of EEG. Journal of the HCI Society of Korea, 325-328.
  4. Xu, H., & Plataniotis, K. N. (2012). Affect recognition using EEG signal. In 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP), 299-304. DOI:10.1109/MMSP.2012.6343458
  5. Jang, J., & Kim, J. (2023). Consistency between individuals of affective responses for multiple modalities based on behavioral and physiological data. Science of Emotion and Sensibility, 26(1), 43-54. DOI: 10.14695/kjsos.2023.26.1.43
  6. Jo, A., Kim, H., & Kim, J. (2023). Affective representation and consistency across individuals responses to affective videos. Science of Emotion and Sensibility. 26(3), 15-28. DOI: 10.14695/kjsos.2023.26.3.15
  7. Kim, H., & Koo. B. (2016). Psychotherapeutic perspective of cerebral lateralization. Korean Psychoanalytic Society, 27(1), 7-16. DOI: 10.18529/psychoanal.2016.27.1.7
  8. Kim, H., & Kim, J. (2022). Affective responses to ASMR using multidimensional scaling and classification. Korean Society for Emotion and Sensibility, 25(3), 47-62. DOI: 10.14695,kjsos.2022.25.3.47 https://doi.org/10.14695,kjsos.2022.25.3.47
  9. Kim, I., & Kim, J. (2024). Affective representation of behavior and physiological responses to emotional videos using wearable devices. Korean Society for Emotion and Sensibility, 27(1), 3-12. DOI: 10.14695/kisos.2024.27.1.1
  10. Kim, J. (2023). Comparison between core affect dimensional structures of different ages using representational similarity analysis. Science of Emotion and Sensibility. 26(1), 33-42. DOI: 10.14695/kjsos.2023.26.1.33
  11. Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: a review. Biological Psychology, 84(3), 394-421. https://doi.org/10.1016/j.biopsycho.2010.03.010
  12. Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis -connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 1-28. DOI: 10.3389/neuro.06.004.2008
  13. Mohamed H., Zaidan B., Zaidan A. A. (2018). A systematic review for human eeg brain signals based emotion classification, feature extraction, brain condition, group comparison. J. Med. Sys, 42(9). DOI: 10.1007/s10916-018-1020-8
  14. Niemic C. (2004). Studies of emotion: a theoretical and empirical review of psychophysiological studies of emotion. J. Underrgr. Res, 1, 15-18.
  15. Park, C., Kim, I., & Kim. J. (2023). Affective representations of basic tastes and intensity using multivariate analyses. Korean Society for Emotion and Sensibility. 26(2), 39-52. DOI: 10.14695/KJSOS.2023.26.2.39
  16. Oude Bos, D. (2006). EEG-based emotion recognition-the influence of visual and auditory stimuli. Capita Selecta (MSc course), 1-17.
  17. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161. DOI:10.1037/h0077714
  18. Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76(5), 805-819. DOI: 10.1037/0022-3514.76.5.805
  19. Saganowski, S., Komoszyńska, J., Behnke, M., Perz, B., Kunc, D., Klich, B., Kaczmarek, Ł. D., & Kazienko, P. (2022). Emognition dataset: emotion recognition with self-reports, facial expressions, and physiology using wearables. Scientific Data, 9(1), 158. DOI: 10.1038/s41597-022-01262-0
  20. Shinkareva, S. V., Wang, J., & Wedell, D. H. (2013). Examining similarity structure: Multidimensional scaling and related approaches in neuroimaging. Computational and Mathematical Methods in Medicine, 2013. DOI: 10.1155/2013/796183
  21. Tran, Y., Craig, A., Boord, P., Connell, K., Cooper, N., Gordon, E., (2006). Personality traits and its association with resting regional brain activity. International Journal of Psychophysiology 60, 215-224. https://doi.org/10.1016/j.ijpsycho.2005.05.008