Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Study on Heart Rate Variability and PSD Analysis of PPG Data for Emotion Recognition

감정 인식을 위한 PPG 데이터의 심박변이도 및 PSD 분석

  • Choi, Jin-young (Department of Computer Science and Engineering, Chungnam National University) ;
  • Kim, Hyung-shin (Department of Computer Science and Engineering, Chungnam National University)
  • 최진영 (충남대학교 컴퓨터공학과) ;
  • 김형신 (충남대학교 컴퓨터공학과)
  • Received : 2017.11.12
  • Accepted : 2018.01.29
  • Published : 2018.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method of recognizing emotions using PPG sensor which measures blood flow according to emotion. From the existing PPG signal, we use a method of determining positive emotions and negative emotions in the frequency domain through PSD (Power Spectrum Density). Based on James R. Russell's two-dimensional prototype model, we classify emotions as joy, sadness, irritability, and calmness and examine their association with the magnitude of energy in the frequency domain. It is significant that this study used the same PPG sensor used in wearable devices to measure the top four kinds of emotions in the frequency domain through image experiments. Through the questionnaire, the accuracy, the immersion level according to the individual, the emotional change, and the biofeedback for the image were collected. The proposed method is expected to be various development such as commercial application service using PPG and mobile application prediction service by merging with context information of existing smart phone.

본 논문에서는 감정에 따라 변화하는 혈류량을 측정하는 PPG 센서를 사용하여 감정을 인식하는 방법을 제안한다. 기존의 PPG 신호로부터 전력스펙트럼밀도(PSD; Power Spectrum Density)를 통해 주파수 영역에서 긍정적인 감정과 부정적인 감정을 판단하는 방법을 활용한다. 제임스 러셀의 이차원 원형 모델에 기반을 두어 감정을 기쁨, 슬픔, 짜증, 평온으로 분류하여 주파수 영역에 따른 에너지의 크기와의 연관성을 살펴본다. 본 연구는 웨어러블 디바이스에서 사용되는 동일한 PPG 센서를 사용하여 상위 네 종류의 감정을 영상 실험을 통해 주파수 영역에서 측정하였다는 것에 의의가 있다. 설문 조사를 통해 정확도와 개인에 따른 몰입 정도와 감정 변화 및 영상에 대한 바이오피드백을 수집하였다. 제안하는 방법은 앞으로 PPG 센서를 사용하는 상용화된 웨어러블 디바이스와 기존에 사용하는 스마트폰의 상황정보와 융합되어 모바일 어플리케이션 예측 서비스 등 다양한 개발이 될 것으로 기대된다.

Keywords

References

  1. W. J. Clancey, Situated Cognition:On Human Knowledge and Computer Representations, Cambridge University, 1997.
  2. D. Yi and C. H. Kah, "Emotions and continued usage of mobile applications," in Industrial Management and Data Systems, Vol. 115, Issue 5, pp. 833-852, 1980. https://doi.org/10.1108/IMDS-11-2014-0338
  3. A. L. Jorge, O. Venet, M. Oscar, F. Mads, B. Jakob, F.-J. Maria and K. V. Lars, "Tell me your apps and I will tell your mood: Correlation of apps usage with Bipolar Disorder State," in Proceedings of the 7th International Conference on Pervasive Technologies Related to Assistive Environments, No. 19, 2014.
  4. T. Yan, C. David, G. Deepak, K. Aman and L. Jie, "Fast App Launching for Mobile Devices Using Predictive User Context," in Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 13-126, 2012.
  5. Veljko Pejovic, et al., "Interrupt Me: Designing Intelligent Prompting Mechanisms for Pervasive Applications," Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp.897-908, Sep 2014.
  6. Jung Chan-Min, et al., "Personalized Application Recommendation System using Emotion Recognition on Smartphone," Korea Institute of Communication Sciences, pp. 236-237, Feb 2012.
  7. CAI Jing, et al., "The Research on Emotion Recognition from ECG Signal," International Conference on Information Technology and Computer Science, Vol. 1, pp. 497-500, Jul 2009.
  8. Bagha, Sangeeta, and Laxmi Shaw. "A real time analysis of PPG signal for measurement of SpO2 and pulse rate." International journal of computer applications, Vol. 36, No. 11, pp. 45-50, Dec 2011.
  9. M. Bolanos, Student Member, IEEE, H. Nazeran, Senior Member, IEEE, and E. Haltiwanger, "Comparison of Heart Rate Variability Signal Features Derived from Electrocardiography and Photoplethysmography in Healthy Individuals." Engineering in Medicine and Biology Society, No. 1, pp. 4289-4294, Sept 2006.
  10. J. Russell, "Pancultural aspects of the human conceptual organization of emotions," Journal of Personality and Social Psychology, Vol. 45(6), pp. 1281-1288, Dec 1983. https://doi.org/10.1037/0022-3514.45.6.1281
  11. Lombardi, Federico, Malliani, Alberto, "Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology," European Heart Journal, Vol. 17, pp. 354-381, March 1996. https://doi.org/10.1093/oxfordjournals.eurheartj.a014868
  12. T. D. M. Trinh and G. P. Daniel, "Where and What Using smartphones to predict next locations and applications in daily life," in Pervasive and Mobile Computing, Vol. 12, pp. 79-91, June 2014. https://doi.org/10.1016/j.pmcj.2013.03.006
  13. S. Akane and P. W. Rosalind, "Stress Recognition using Wearable Sensors and Mobile Phones," in Humanize Association Conference on Affective Computing and Intelligent Interaction, pp. 671-676, Sep 2013
  14. P. Martin, D. Tilman, P. S. Jose and O. Nuria, "When Attention is not Scarce - Detecting Boredom from Mobile Phone Usage," in Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 825-836, Sep 2015.
  15. K. Rachuri, M. Musolesi, C. Mascolo, P. Rentfrow, C. Longworth, A. Aucinas, "EmotionSense: a mobile phones based adaptive platform for experimental social psychology research," in Proceedings of the 12th ACM international conference on Ubiquitous computing, pp. 281-290, Sep 2010.
  16. L. Robert, L. Yunxin, L. D. Nicholas and Z. Lin, "MoodSchope: Building a Mood Sensor from Smartphone Usage Patterns," in Proceedings of the 11th annual international conference on Mobile systems, applications, and services, pp. 389-402, Sep 2013.
  17. H.W. Kim, W.J. Choi, S.Y. Ihm, Y.H. Park, "A Study on Utilizing a Vital Sign for Emotional Objectification," The Journal of Digital Contents Society, Vol. 18, No. 3, pp. 485-497, June 2017. https://doi.org/10.9728/DCS.2017.18.3.485
  18. S. Kay and S. Marple, "Spectrum analysis-A modern Perspective," in Proceedings of the IEEE, Vol. 69, No. 11, pp. 1380-1419, Nov 1981. https://doi.org/10.1109/PROC.1981.12184
  19. W. J. G., Design of Pulse Oximeter, in Institute of physics Publishing, 1997.
  20. S.H. Hong, M.J. Bae, "Sound of Improving One's Ability to Concentrate," in Proceedings of the Institute of Electronics and Information Engineers, pp. 671-672, Nov 2007.