Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Emotion Classification based on EEG signals with LSTM deep learning method

어텐션 메커니즘 기반 Long-Short Term Memory Network를 이용한 EEG 신호 기반의 감정 분류 기법

  • Received : 2020.09.02
  • Accepted : 2020.12.29
  • Published : 2021.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

This study proposed a Long-Short Term Memory network to consider changes in emotion over time, and applied an attention mechanism to give weights to the emotion states that appear at specific moments. We used 32 channel EEG data from DEAP database. A 2-level classification (Low and High) experiment and a 3-level classification experiment (Low, Middle, and High) were performed on Valence and Arousal emotion model. As a result, accuracy of the 2-level classification experiment was 90.1% for Valence and 88.1% for Arousal. The accuracy of 3-level classification was 83.5% for Valence and 82.5% for Arousal.

본 연구에서는 EEG 신호를 기반으로 감정 인식에 유용한 딥러닝 기법을 제안한다. 감정이 시간에 따라 변화하는 특성을 반영하기 위해 Long-Short Term Memory 네트워크를 사용하였다. 또한, 특정 시점의 감정적 상태가 전체 감정 상태에 영향을 미친다는 이론을 기반으로 특정 순간의 감정 상태에 가중치를 주기 위해 어텐션 메커니즘을 적용했다. EEG 신호는 DEAP 데이터베이스를 사용하였으며, 감정은 긍정과 부정의 정도를 나타내는 정서가(Valence)와 감정의 정도를 나타내는 각성(Arousal) 모델을 사용하였다. 실험 결과 정서가(Valence)와 각성(Arousal)을 2단계(낮음, 높음)로 나누었을 때 분석 정확도는 정서가(Valence)의 경우 90.1%, 각성(Arousal)의 경우 88.1%이다. 낮음, 중간, 높음의 3단계로 감정을 구분한 경우 정서가(Valence)는 83.5%, 각성(Arousal)은 82.5%의 정확도를 보였다.

Keywords

References

  1. Al-Fahoum, S., and Al-Fraihat, A. (2014). Methods of EEG Signal Features Extraction Using Linear Analysis in Frequency and Time-Frequency Domains. ISRN Neuroscience, 2014, 1-7. https://doi.org/10.1155/2014/730218
  2. Al-Nafjan, A., Hosny, M., Al-Wabil, A., and Al-Ohali, Y. (2017). Classification of Human Emotions from Electroencephalogram (EEG) Signal using Deep Neural Network. International Journal of Advanced Computer Science and Applications, 8(9), 419-425. https://doi.org/10.14569/ijacsa.2017.080955
  3. Alhagry, S., Aly, A., and A., R. (2017). Emotion Recognition based on EEG using LSTM Recurrent Neural Network. International Journal of Advanced Computer Science and Applications, 8(10). https://doi.org/10.14569/ijacsa.2017.081046
  4. Atkinson, J., and Campos, D. (2016). Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Systems with Applications, 47, 35-41. https://doi.org/10.1016/j.eswa.2015.10.049
  5. Chung, S. Y., and Yoon, H. J. (2012). Affective classification using Bayesian classifier and supervised learning. International Conference on Control, Automation and Systems, 1768-1771.
  6. Fredrickson, B. L., and Kahneman, D. (1993). Duration Neglect in Retrospective Evaluations of Affective Episodes. Journal of Personality and Social Psychology, 65(1), 45-55. https://doi.org/10.1037/0022-3514.65.1.45
  7. Hemanth, D. J., Anitha, J., and Son, L. H. (2018). Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks. Computers and Electrical Engineering, 68, 170-180. https://doi.org/10.1016/j.compeleceng.2018.04.006
  8. Hochreiter, S., and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  9. Huang, X., Kortelainen, J., Zhao, G., Li, X., Moilanen, A., Seppanen, T., and Pietikainen, M. (2016). Multi-modal emotion analysis from facial expressions and electroencephalogram. Computer Vision and Image Understanding, 147, 114-124. https://doi.org/10.1016/j.cviu.2015.09.015
  10. Jirayucharoensak, S., Pan-Ngum, S., and Israsena, P. (2014). EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation. Scientific World Journal, 2014. https://doi.org/10.1155/2014/627892
  11. Koelstra, S., Mühl, C., Soleymani, M., Lee, J. S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., and Patras, I. (2012). DEAP: A database for emotion analysis; Using physiological signals. IEEE Transactions on Affective Computing, 3(1), 18-31. https://doi.org/10.1109/T-AFFC.2011.15
  12. Mark L. Howe (2011). The Nature of Ealry Memory: An Adaptive Theory of the Genesis and Development of Memory, Oxford University Press.
  13. Posner, J., Russell, J. A., and Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17(3), 715-734. https://doi.org/10.1017/S0954579405050340
  14. World Health Organization (2020). Depression, https://www.who.int/news-room/fact-sheets/detail/depression(Accessed on Jul. 16th, 2020)
  15. Zhang, X., Hu, B., Chen, J., and Moore, P. (2013). Ontology-based context modeling for emotion recognition in an intelligent web. World Wide Web, 16(4), 497-513. https://doi.org/10.1007/s11280-012-0181-5