DOI QR코드

DOI QR Code

Real-time Vital Signs Measurement System using Facial Image Data

안면 이미지 데이터를 이용한 실시간 생체징후 측정시스템

  • Received : 2020.10.30
  • Accepted : 2021.03.09
  • Published : 2021.03.30

Abstract

The purpose of this study is to present an effective methodology that can measure heart rate, heart rate variability, oxygen saturation, respiration rate, mental stress level, and blood pressure using mobile front camera that can be accessed most in real life. Face recognition was performed in real-time using Blaze Face to acquire facial image data, and the forehead was designated as ROI (Region Of Interest) using feature points of the eyes, nose, and mouth, and ears. Representative values for each channel of the ROI were generated and aligned on the time axis to measure vital signs. The vital signs measurement method was based on Fourier transform, and noise was removed and filtered according to the desired vital signs to increase the accuracy of the measurement. To verify the results, vital signs measured using facial image data were compared with pulse oximeter contact sensor, and TI non-contact sensor. As a result of this work, the possibility of extracting a total of six vital signs (heart rate, heart rate variability, oxygen saturation, respiratory rate, stress, and blood pressure) was confirmed through facial images.

본 연구는 실생활에서 가장 많이 접할 수 있는 모바일 전면 카메라를 이용하여 심장박동, 심장박동 변이율, 산소포화도, 호흡도, 스트레스수치, 혈압을 측정할 수 있는 효과적인 방법론을 제시하는 것이 목적이다. Blaze Face를 이용하여 실시간으로 얼굴인식을 진행하여 안면 이미지 데이터를 취득하고 눈, 코 입, 귀의 특징 점을 이용하여 이마를 관심영역으로 지정하며 평균값을 시간 축으로 정렬한 후 생체징후 측정에 이용하였다. 생체징후 측정 기법은 fourier transform을 기본으로 이용하였으며, 측정하고자 하는 생체징후에 맞게 노이즈 제거 및 필터 처리함으로써 측정값의 정확도를 향상 시켰다. 결과를 검증하기 위해 접촉식 센서와 비접촉식 센서 비교를 진행하였다. 분석 결과 안면 이미지를 이용하여 심장박동, 심장 박동 변이율, 산소포화도, 호흡도, 스트레스, 혈압 총 여섯 가지 생체 징후를 추출 할 수 있는 가능성을 확인하였다.

Keywords

References

  1. National Early Warning Score(NEWS), https://www.mdcalc.com/national-early-warning-score-news
  2. J. Kim, K. Lee, "A Comparative Study on the Optimal Model for abnormal Detection event of Heart Rate Time Series Data Based on the Correlation between PPG and ECG", https://doi.org/10.7472/jksii.2019.20.6.137 (accessed Aug, 30, 2019)
  3. V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, M. Grundmann, "BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs", CVPR Workshop on Computer Vision for Augmented and Virtual Reality, 2019.
  4. V. Kublanov, K. Purtov and D. Belkov, "Remote Photoplethysmography for the Neuro-electrostimulation Procedures Monitoring The Possibilities of Remote Photoplethysmography Application for the Analysis of High Frequency Parameters of Heart Rate Variability." 10th International Conference on Bio-Inspired Systems and Signal Processing, BIOSIGNALS 2017-Part of 10th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2017. SciTePress, 2017.
  5. Luchini, D. Paulo, et al,"Validation of a spectrophotometric method for quantification of carboxyhemoglobin." Journal of analytical toxicology Vol.33 No.8, pp.540-544. 2019. https://doi.org/10.1093/jat/33.8.540
  6. Fred.S, J.P. Ginsberg,"An Overview of Heart Rate Variability Metrics and Norms", Frontiers in public health 5 ,2017
  7. Delaney, J.P.A, & Brodie, D.A,"Effects of short-term psychological stress on the time and frequency domains of heart-rate variability." Perceptual and motor skills Vol.91 No.2 , pp.515-524, 2000. https://doi.org/10.2466/pms.2000.91.2.515
  8. Sanyal, Shourjya, and K. K.Nundy. "Algorithms for monitoring heart rate and respiratory rate from the video of a user's face." IEEE Journal of translational engineering in health and medicine Vol.6, pp.1-11. 2018. https://doi.org/10.1109/JTEHM.2018.2818687
  9. Scholander, P. F. "Oxygen transport through hemoglobin solutions." Science Vol.131 No.3400, pp.585-590,1960 https://doi.org/10.1126/science.131.3400.585
  10. Suna, Gurhan, and M. Alp. "Comparison of Strength, Heart Rate, Oxygen Saturation and Technical Test Values of 12-14 Year Male Tennis Players in Competition Period." Journal of Education and Learning Vol.8 No.6, pp.187-194, 2019. https://doi.org/10.5539/jel.v8n6p187
  11. mCube-HealthCare IOT Device, http://www.cndi.co.kr/cndi/board/bbs/board.php?bo_table=flan&wr_id=5
  12. E. Cha, K. Jong, Development of an IoB-Based HW/SW Platform for Human Motion Detection and Heart Rate Measurement . Journal of Broadcast Engineering, pp.178-180, Nov 2019
  13. D. Kim, J. Kim, K. Kwang, Real-time vital signs measurement system using facial image on Mobile, Journal of Broadcast Engineering, pp.94-97, Nov 2020. https://doi.org/10.5909/JBE.2020.25.1.94