Browse > Article
http://dx.doi.org/10.14695/KJSOS.2018.21.1.177

Classification of Negative Emotions based on Arousal Score and Physiological Signals using Neural Network  

Kim, Ahyoung (Bio-Medical IT Convergence Research Division, Electronics and Telecommunications Research Institute)
Jang, Eun-Hye (Bio-Medical IT Convergence Research Division, Electronics and Telecommunications Research Institute)
Sohn, Jin-Hun (Department of Psychology, Brain Research Institute, Chungnam National University)
Publication Information
Science of Emotion and Sensibility / v.21, no.1, 2018 , pp. 177-186 More about this Journal
Abstract
The mechanism of emotion is complex and influenced by a variety of factors, so that it is crucial to analyze emotion in broad and diversified perspectives. In this study, we classified neutral and negative emotions(sadness, fear, surprise) using arousal evaluation, which is one of the psychological evaluation scales, as well as physiological signals. We have not only revealed the difference between physiological signals coupled to the emotions, but also assessed how accurate these emotions can be classified by our emotional recognizer based on neural network algorithm. A total of 146 participants(mean age $20.1{\pm}4.0$, male 41%) were emotionally stimulated while their physiological signals of the electrocardiogram, blood flow, and dermal activity were recorded. In addition, the participants evaluated their psychological states on the emotional rating scale in response to the emotional stimuli. Heart rate(HR), standard deviation(SDNN), blood flow(BVP), pulse wave transmission time(PTT), skin conduction level(SCL) and skin conduction response(SCR) were calculated before and after the emotional stimulation. As a result, the difference between physiological responses was verified corresponding to the emotions, and the highest emotion classification performance of 86.9% was obtained using the combined analysis of arousal and physiological features. This study suggests that negative emotion can be categorized by psychological and physiological evaluation along with the application of machine learning algorithm, which can contribute to the science and technology of detecting human emotion.
Keywords
Emotion Classification; Negative Emotion; Neural Network; Arousal; Physiological Signals;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Badcock, N. A., Preece, K. A., de Wit, B., Glenn, K., Fieder, N., Thie, J., & McArthur, G. (2015). Validation of the Emotiv EPOC EEG system for research quality auditory event-related potentials in children. PeerJ, 3, e907. DOI: 10.7717/ peerj.907   DOI
2 Banziger, T., Grandjean, D., & Scherer, K. R. (2009). Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). Emotion (Washington, D.C.), 9(5), 691-704. DOI: 10.1037/a0017088   DOI
3 Basu, S., Jana, N., Bag, A. M. M., Mukherjee, J., Kumar, S., & Guha, R. (2015). Emotion recognition based on physiological signals using valence-arousal model. In 2015 Third International Conference on Image Information Processing(ICIIP)(pp. 50-55). DOI: 10.1109/ICIIP.2015.7414739
4 Calvo, R. A., & D'Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18-37. DOI: 10.1109/T-AFFC.2010.1   DOI
5 Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001). Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, 18(1), 32-80. DOI: 10.1109/79.911197   DOI
6 Duan, R. N., Zhu, J. Y., & Lu, B. L. (2013). Differential entropy feature for EEG-based emotion classification. In 2013 6th International IEEE/EMBS Conference on Neural Engineering(NER) (pp. 81-84). DOI: 10.1109/NER.2013.6695876
7 Ebner, N. C., & Fischer, H. (2014). Emotion and aging: evidence from brain and behavior. Frontiers in Psychology, 5. DOI: 10.3389/fpsyg.2014.00996
8 Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6, 169-200. DOI: 10.1080/02699939208411068   DOI
9 Jirayucharoensak, S., Pan-Ngum, S., & Israsena, P. (2014). EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. The Scientific World Journal, 627892. DOI: 10.1155/2014/627892
10 Jang, E.-H., Park, B.-J., Park, M.-S., Kim, S.-H., & Sohn, J.-H. (2015). Analysis of physiological signals for recognition of boredom, pain, and surprise emotions. Journal of Physiological Anthropology, 34(1), 25. DOI: 10.1186/s40101-015-0063-5   DOI
11 Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: a review. Biological Psychology, 84(3), 394-421. DOI: 10.1016/j.biopsycho.2010.03.010   DOI
12 Majumder, S., Mondal, T., & Deen, M. J. (2017). Wearable sensors for remote health monitoring. Sensors (Basel, Switzerland), 17(1). DOI: 10.3390/s17010130
13 Levenson, R. W., Ekman, P., & Friesen, W. V. (1990). Voluntary facial action generates emotion-specific autonomic nervous system activity. Psychophysiology, 27(4), 363-384. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/2236440   DOI
14 Lin, Y. P., Wang, C. H., Wu, T. L., Jeng, S. K., & Chen, J. H. (2009). EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 489-492). DOI: 10.1109/ICASSP.2009.4959627
15 Lindquist, K. A., MacCormack, J. K., & Shablack, H. (2015). The role of language in emotion: Predictions from psychological constructionism. Frontiers in Psychology, 6, 444. DOI: 10.3389/fpsyg.2015.00444
16 Mill, A., Allik, J., Realo, A., & Valk, R. (2009). Agerelated differences in emotion recognition ability: a cross-sectional study. Emotion (Washington, D.C.), 9(5), 619-630. DOI: 10.1037/a0016562   DOI
17 Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161-1178. DOI: 10.1037/h0077714   DOI
18 Nasoz, F., Alvarez, K., Lisetti, C. L., & Finkelstein, N. (2004). Emotion recognition from physiological signals using wireless sensors for presence technologies. Cognition, Technology & Work, 6(1), 4-14. DOI: 10.1007/s10111-003-0143-x   DOI
19 Picard, R. W. (2003). Affective computing: challenges. International Journal of Human-Computer Studies, 59(1), 55-64. DOI: 10.1016/S1071-5819(03)00052-1   DOI
20 Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), 1175-1191. DOI: 10.1109/34.954607   DOI
21 Simon, E. W., Rosen, M., Grossman, E., & Pratowski, E. (1995). The relationships among facial emotion recognition, social skills, and quality of life. Research in Developmental Disabilities, 16(5), 383-391. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/8532917   DOI
22 Tan, J.-W., Andrade, A. O., Li, H., Walter, S., Hrabal, D., Rukavina, S., Limbrecht-Ecklundt, K., Hoffman, H., Traue, H. C. (2016). Recognition of intensive valence and arousal affective states via facial electromyographic activity in young and senior adults. PLoS ONE, 11(1), e0146691. DOI: 10.1371/journal.pone.0146691   DOI
23 Wagner, J., Kim, J., & Andre, E. (2005). From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In 2005 IEEE International Conference on Multimedia and Expo(pp. 940-943). DOI: 10.1109/ICME.2005.1521579
24 Wiem, M. B. H., & Lachiri, Z. (2016). Emotion assessing using valence-arousal evaluation based on peripheral physiological signals and support vector machine. In 2016 4th International Conference on Control Engineering Information Technology(CEIT) (pp. 1-5). DOI: 10.1109/CEIT.2016.7929117
25 Lee, K. H. (1997). Human sensibility and its measurement and evaluation. Annual Conference Papers of Korean Society for Emotion & Sensibility (pp. 37-42). Daejeon, Republic of Korea. Retrieved from http://www.koses.or.kr/
26 Zaja, R. H., & Rojahn, J. (2008). Facial emotion recognition in intellectual disabilities. Current Opinion in Psychiatry, 21(5), 441-444. DOI: 10.1097/YCO.0b013e328305e5fd   DOI
27 Zhang, Q., Chen, X., Zhan, Q., Yang, T., & Xia, S. (2017). Respiration-based emotion recognition with deep learning. Computers in Industry, 92-93, 84-90. DOI: 10.1016/j.compind.2017.04.005   DOI
28 Park, M. S., Kim, H. E., & Sohn, J. H. (2011). Development of emotion-evoking stimuli to provoke spontaneous emotions. Proceedings for the 2011 Annual Spring Conference of Korean Society for Emotion & Sensibility (pp. 505-512). Daejeon, Republic of Korea. Retrieved from http://www.koses.or.kr/