Browse > Article
http://dx.doi.org/10.15207/JKCS.2018.9.5.033

Convergence Implementing Emotion Prediction Neural Network Based on Heart Rate Variability (HRV)  

Park, Sung Soo (SKK Business School, Sungkyunkwan University)
Lee, Kun Chang (SKK Business School, Sungkyunkwan University)
Publication Information
Journal of the Korea Convergence Society / v.9, no.5, 2018 , pp. 33-41 More about this Journal
Abstract
The purpose of this study is to develop more accurate and robust emotion prediction neural network (EPNN) model by combining heart rate variability (HRV) and neural network. For the sake of improving the prediction performance more reliably, the proposed EPNN model is based on various types of activation functions like hyperbolic tangent, linear, and Gaussian functions, all of which are embedded in hidden nodes to improve its performance. In order to verify the validity of the proposed EPNN model, a number of HRV metrics were calculated from 20 valid and qualified participants whose emotions were induced by using money game. To add more rigor to the experiment, the participants' valence and arousal were checked and used as output node of the EPNN. The experiment results reveal that the F-Measure for Valence and Arousal is 80% and 95%, respectively, proving that the EPNN yields very robust and well-balanced performance. The EPNN performance was compared with competing models like neural network, logistic regression, support vector machine, and random forest. The EPNN was more accurate and reliable than those of the competing models. The results of this study can be effectively applied to many types of wearable computing devices when ubiquitous digital health environment becomes feasible and permeating into our everyday lives.
Keywords
convergence emotion prediction; heart rate variability; HRV; artificial neural network; activation function;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 T. Kageyama, N. Nishikido, T. Kobayashi, Y. Kurokawa, T. Kaneko & M. Kabuto. (1998). Self-Reported Sleep Quality, Job Stress, and Daytime Autonomic Activities Assessed in Terms of Short-Term Heart Rate Variability among Male White-Collar Workers. Industrial Health, 36(3), 263-272. DOI : 10.2486/indhealth.36.263   DOI
2 T. G. M. Vrijkotte, L. J. P. van Doornen & E. J. C. de Geus. (2000). Effects of Work Stress on Ambulatory Blood Pressure, Heart Rate, and Heart Rate Variability. Hypertension, 35, 880-886. DOI : 10.1161/01.HYP.35.4.880   DOI
3 C. M. Bishop. (1995). Neural networks for pattern recognition. Oxford university press.
4 N. Kandasamy, S. N. Garfinkel, L. Page, B. Hardy, H. D. Critchley, M. urnell & J. M. Coates. (2016). Interoceptive ability predicts survival on a London trading floor, Scientific Reports, 6, 32986. DOI : 10.1038/srep32986   DOI
5 R. W. Picard, E. Vyzas & J. Healey. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE transactions on pattern analysis and machine intelligence, 23(10), 1175-1191. DOI : 10.1109/34.954607   DOI
6 I. K. Kwon, S. Y. Lee. (2012). Personalized Service Based on Context Awareness through User Emotional Perception in Mobile Environment. Journal of Digital Convergence, 10(2), 287-292.   DOI
7 K. J. Kim. (2018). A conversation training program that combines reason and sensitivity - Using the P-A-C technique of Transactional analysis -. Journal of Convergence for Information Technology, 8(2), 149-155.   DOI
8 F. Agrafioti, D. Hatzinakos & A. K. Anderson. (2012). ECG pattern analysis for emotion detection. IEEE Transactions on Affective Computing, 3(1), 102-115. DOI : 10.1109/T-AFFC.2011.28   DOI
9 A. Hariharan & M. T. P. Adam. (2015). Blended emotion detection for decision support. IEEE Transactions on Human-Machine Systems, 45(4,), 510-517. DOI : 10.1109/THMS.2015.2418231   DOI
10 G. Valenza, L. Citi, A. Lanata, E. P. Scilingo & R. Barbieri. (2014). Revealing real-time emotional responses: a personalized assessment based on heartbeat dynamics. Scientific reports, 4, 4998. DOI : 10.1038/srep04998
11 S. S. Park & K. C. Lee. (2017). Emotion prediction neural network to understand how emotion is predicted by using heart rate variability measurements. Journal of The Korea Society of Computer and Information, 22(7), 75-82. DOI: 10.9708/jksci.2017.22.07.075   DOI
12 J. Kim. (2016). Emotion Prediction of Paragraph using Big Data Analysis. Journal of Digital Convergence, 14(11), 267-273. DOI : 10.14400/JDC.2016.14.11.267   DOI
13 M. Dorofki, A. H. Elshafie, O. Jaafar, O. A. Karim & S. Mastura. (2012). Comparison of artificial neural network transfer functions abilities to simulate extreme runoff data. International Proceedings of Chemical, Biological and Environmental Engineering, 33, 39-44.
14 R. R. Singh, S. Conjeti & R. Banerjee. (2013). A comparative evaluation of neural network classifiers for stress level analysis of automotive drivers using physiological signals. Biomedical Signal Processing and Control, 8(6), 740-754. DOI : 10.1016/j.bspc.2013.06.014   DOI
15 M. A. Nicolaou, H. Gunes & M. Pantic. (2011). Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Transactions on Affective Computing, 2(2), 92-105. DOI : 10.1109/T-AFFC.2011.9   DOI
16 F. Ringeval, F. Eyben, E. Kroupi, A. Yuce, J. P. Thiran, T. Ebrahimi, D. Lalanne & B. Schuller. (2015). Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data. Pattern Recognition Letters, 66, 22-30. DOI : 10.1016/j.patrec.2014.11.007   DOI
17 S. S. Park & K. C. Lee. (2018). Analysis of the Relative Importance of HRV Metrics to Predict Emotion by Using Valence-Arousal driven Neural Network. under review in Journal of Korean Institute of Information Technology.
18 M. D. Asaduzzaman, M. Shahjahan & K. Murase. (2009). Faster training using fusion of activation functions for feed forward neural networks. International journal of neural systems, 19(6), 437-448. DOI : 10.1142/S0129065709002130   DOI
19 M. J. Song, M. Y. Kim, I. S. Sim & W. S. Kim. (2010). Evaluation of Horticultural Therapy on the Emotional Improvement of Depressed Patients by Using Heart Rate Variability. Korean Journal of Horticultural Science & Technology, 28(6), 1066-1071.
20 M. S. Kim, Y. N. Kim & Y. C. Cho. (2015). Electrocardiographic characteristics of significant factors of detected atrial fibrillation using WEMS. Journal of the Korea Industrial Information Systems Research, 20(6), 37-46. DOI : 10.9723/jksiis.2015.20.6.037   DOI
21 M. Sansone, R. Fusco, A. Pepino & C. Sansone. (2013). Electrocardiogram pattern recognition and analysis based on artificial neural networks and support vector machines: A review. Journal of Healthcare Engineering, 4(4), 465-504. DOI : 10.1260/2040-2295.4.4.465   DOI
22 N. Ravaja. (2014). Contributions of psychophysiology to media research: Review and recommendations. Media Psychology, 6(2), 193-235. DOI : 10.1207/s1532785xmep0602_4   DOI
23 M. Sokolova & G. Lapalme. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427-437. DOI : 10.1016/j.ipm.2009.03.002   DOI
24 H. Hwang & S. Park. (2011). The Impact of User's Psychological Experience on Online Game Addiction: Perceived Reality and Sense of Presence. Journal of communication science, 11(1), 471-505.
25 J. Posner, J. Russell & B. Peterson, (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and psychopathology, 17(3), 715-734. DOI : 10.1017/S0954579405050340   DOI
26 M. P. Tarvainen, J. P. Niskanen, J. A. Lipponen, P. O. Ranta-Aho & P. A. Karjalainen. (2014). Kubios HRV - heart rate variability analysis software. Computer methods and programs in biomedicine, 113(1), 210-220. DOI : 10.1016/j.cmpb.2013.07.024   DOI
27 S. N. Yu & S. F. Chen. (2015). Emotion state identification based on heart rate variability and genetic algorithm. Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE, 538-541. DOI : 10.1109/EMBC.2015.7318418
28 R. Hecht-Nielsen. (1989). Theory of the Back Propagation Neural Network. In Proceeding of the International Joint Conference on Neural Networks(IJCNN), New York, 593-605. DOI : 10.1016/B978-0-12-741252-8.50010-8
29 S. M. Ahn, M. C. Whang, D K. Kim, J H. Kim & S. I. Park. (2012). Real-time emotion recognition technology using individualization processemotional technology. Korean Journal of the Science of Emotion and Sensibility, 15(1), 133-140.
30 M. M. Bradley & P. J. Lang. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry, 25(1), 49-59. DOI : 10.1016/0005-7916(94)90063-9   DOI
31 U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim & J. S. Suri. (2006). Heart rate variability: a review. Medical and biological engineering and computing, 44(12), 1031-1051. DOI : 10.1007/s11517-006-0119-0   DOI
32 D. Kukolja, S. Popovi, M. Horvat, B. Kova & K. osi. (2014). Comparative analysis of emotion estimation methods based on physiological measurements for real-time applications. International Journal of Human-Computer Studies, 72(10), 717-727. DOI : 10.1016/j.ijhcs.2014.05.006   DOI
33 A. Haag, S. Goronzy, P. Schaich & J. Williams. (2004). Emotion recognition using bio-sensors: First steps towards an automatic system. Affective dialogue systems, Springer, 36-48.
34 M. Swangnetr & D. B. Kaber. (2013). Emotional state classification in patient-robot interaction using wavelet analysis and statistics-based feature selection. IEEE Transactions on Human-Machine Systems, 43(1), 63-75. DOI : 10.1109/TSMCA.2012.2210408   DOI
35 F. A. Russo, N. N. Vempala & G. M. Sandstrom. (2013). Predicting musically induced emotions from physiological inputs: Linear and neural network models. Frontiers in Psychology, 4-468. DOI : 10.3389/fpsyg.2013.00468