Journal of Advanced Navigation Technology (한국항행학회논문지)

The Korean Navigation Institute (한국항행학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Interactive Content Services through an Intelligent IoT Mirror System

지능형 IoT 미러 시스템을 활용한 인터랙티브 콘텐츠 서비스 구현

  • Jung, Wonseok (Department of Information and Communication Engineering, Namseoul University) ;
  • Seo, Jeongwook (Department of Information and Communication Engineering, Namseoul University)
  • 정원석 (남서울대학교 정보통신공학과) ;
  • 서정욱 (남서울대학교 정보통신공학과)
  • Received : 2018.09.13
  • Accepted : 2018.10.22
  • Published : 2018.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we develop interactive content services for preventing depression of users through an intelligent Internet of Things(IoT) mirror system. For interactive content services, an IoT mirror device measures attention and meditation data from an EEG headset device and also measures facial expression data such as "sad", "angery", "disgust", "neutral", " happy", and "surprise" classified by a multi-layer perceptron algorithm through an webcam. Then, it sends the measured data to an oneM2M-compliant IoT server. Based on the collected data in the IoT server, a machine learning model is built to classify three levels of depression (RED, YELLOW, and GREEN) given by a proposed merge labeling method. It was verified that the k-nearest neighbor (k-NN) model could achieve about 93% of accuracy by experimental results. In addition, according to the classified level, a social network service agent sent a corresponding alert message to the family, friends and social workers. Thus, we were able to provide an interactive content service between users and caregivers.

본 논문에서는 지능형 IoT (internet of things) 미러 시스템을 통해 사용자의 우울증 예방을 위한 인터랙티브 콘텐츠 서비스를 구현한다. 인터랙티브 콘텐츠 서비스를 위해 IoT 미러 장치는 뇌파 헤드셋 디바이스로부터 집중도 및 명상도 데이터를 측정하고, 웹캠을 통해 다층 퍼셉트론 알고리즘으로 분류된 "슬픔", "분노", "혐오감", "중립", "행복" 및 "놀람"과 같은 표정 데이터를 측정한 후, oneM2M 표준을 준용한 IoT 서버로 전송한다. IoT 서버에 수집된 데이터는 제안한 병합 레이블링 과정을 거쳐 세 가지의 우울 단계(RED, YELLOW, GREEN)를 분류하는 기계학습 모델을 생성한다. 실험을 통해 k-최근접 이웃 모델로 우울 단계를 분류한 결과 약 93%의 정확도를 얻을 수 있었고, 분류된 우울 단계에 따라 가족, 친구 및 사회복지사에게 소셜 네트워크 서비스 에이전트를 통해 알림 메시지를 전송하여 사용자와 보호자 간의 인터랙티브 콘텐츠 서비스를 구현하였다.

Keywords

References

  1. B. W. Kang, M. H. Jung and B. J. Cho, “The effects of psychological factorson elderly of depression scale and electroephalography,” Journal of the Korea Society of Computer and Information, Vol. 21, No. 12, pp. 157-163, Dec. 2016. https://doi.org/10.9708/JKSCI.2016.21.12.157
  2. J. U. Ko and S. B. Kim, "A study of influence factors on the suicidal tendency of elderly living alone," The Journal of Korean Academy of Nursing, Vol. 30, pp. 29-48, Dec. 2011.
  3. Korea Centers for Disease Control and Prevention (KCDC). Emergency room injury patient specimen in-depth investigation [Internet]. Available: http://www.cdc.go.kr/CDC/intro/CdcKrIntro0201.jsp?menuIds=HOME001-MNU1154-MNU0005-MNU0011&cid=8803/.
  4. C. G. Kim and S. M. Park, “Gender difference in risk factors for depression in community-dwelling elders,” The Journal of Korean Academy of Nursing, Vol. 42, No. 1, pp. 136-147, Feb. 2012. https://doi.org/10.4040/jkan.2012.42.1.136
  5. E. R. Lee, J. H. Kang and J. P. Jung, “Factors influencing the depression of aged people,” Journal of The Korea Contents Society, Vol. 13, No. 7, pp. 290-300, Jul. 2013.
  6. J. G. Woo, S. M. Ji, S. G. Kim, K. D. Woo, H. C. Choi, J. h. Goo, J. J. Lee, C. W. Lee and S. W. Baik, "The development of intuitive authoring tool for interactive contents," in Proceedings of the Korean Insitute of Information Scientists and Engineers, Jeju: Korea, pp. 204-209, Jun. 2009.
  7. J. Y. Han, “Possibilities in application of art therapy employing digital contents with play-instinct of the art,” The Journal of Korean Society of Media & Arts, Vol. 13, No. 3, pp. 145-160, Jun. 2015.
  8. Raspberry Pi. Documentation [Internet]. Available: https://www.raspberrypi.org/documentation/hardware/
  9. Google. Googlecloudspeech-to-textdocumentation [Internet]. Available: https://cloud.google.com/speech/docs/?hl=ko/.
  10. M. Beyeler, OpenCV with python blueprints, Birmingham, Packt Publishing, 2015.
  11. D. I. Singh, “Face recognition through multilayer perceptron (MLP) and learning vector quantization (LVQ),” International The Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE), Vol. 1, No. 10, pp. 87-90, Dec. 2012.
  12. NeuroSky, MindWave mobile: user guide, 2017.
  13. oneM2M, Functional architecture, TS-0001-v2.10.0, 2016.
  14. OCEAN, nCube-Thyme(Node.js) developer guide, v1.0, 2017.
  15. OCEAN, Mobius release 2 installation guide, v2.0.0, 2017.
  16. A. C. Muller and S. Guido, Introdution to machine learning with python, O'Reilly Media, 2016.