Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

The Edge Computing System for the Detection of Water Usage Activities with Sound Classification

음향 기반 물 사용 활동 감지용 엣지 컴퓨팅 시스템

  • Seung-Ho Hyun (School of Electrical Engineering, University of Ulsan) ;
  • Youngjoon Chee (School of Electrical Engineering, University of Ulsan)
  • 현승호 (울산대학교 공과대학 전기공학부) ;
  • 지영준 (울산대학교 공과대학 전기공학부)
  • Received : 2023.02.07
  • Accepted : 2023.04.23
  • Published : 2023.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Efforts to employ smart home sensors to monitor the indoor activities of elderly single residents have been made to assess the feasibility of a safe and healthy lifestyle. However, the bathroom remains an area of blind spot. In this study, we have developed and evaluated a new edge computer device that can automatically detect water usage activities in the bathroom and record the activity log on a cloud server. Three kinds of sound as flushing, showering, and washing using wash basin generated during water usage were recorded and cut into 1-second scenes. These sound clips were then converted into a 2-dimensional image using MEL-spectrogram. Sound data augmentation techniques were adopted to obtain better learning effect from smaller number of data sets. These techniques, some of which are applied in time domain and others in frequency domain, increased the number of training data set by 30 times. A deep learning model, called CRNN, combining Convolutional Neural Network and Recurrent Neural Network was employed. The edge device was implemented using Raspberry Pi 4 and was equipped with a condenser microphone and amplifier to run the pre-trained model in real-time. The detected activities were recorded as text-based activity logs on a Firebase server. Performance was evaluated in two bathrooms for the three water usage activities, resulting in an accuracy of 96.1% and 88.2%, and F1 Score of 96.1% and 87.8%, respectively. Most of the classification errors were observed in the water sound from washing. In conclusion, this system demonstrates the potential for use in recording the activities as a lifelog of elderly single residents to a cloud server over the long-term.

Keywords

Acknowledgement

본 연구는 산업통상자원부와 한국산업기술진흥원의 "국제공동기술개발사업"의 지원을 받아 수행된 연구결과임(과제번호: P088100009).

References

  1. https://www.index.go.kr/unify/idx-info.do?idxCd=4233. Accessed on 5th Feb, 2023, K-indicator, National Index System. 
  2. Moon HJ, Kim MH. Housing Circumstances of the Elderly Households Who Live Alone or/with Spouse. Journal of Korean Housing Association. 2004;15(1):75-84. 
  3. Storeng SH, Sund ER, Krokstad S. Factors associated with basic and instrumental activities of daily living in elderly participants of a population-based survey: the Nord-Trondelag Health Study. BMJ Open 2018;8(3):e018942. 
  4. Shin YS, Kim WS, Shin IS, Kim GH, Jung DY. A Comparative Study between Physical, Cognitive and Emotional State on Living Activity Pattern of Elderly People. Korean Journal of Gerontological Social Welfare. 2017;72(4):7-29.  https://doi.org/10.21194/KJGSW.72.4.201712.7
  5. Baig M, Afifi S, GholamHosseini H, Mirza F. A Systematic Review of Wearable Sensors and IoT-Based Monitoring Applications for Older Adults - a Focus on Ageing Population and Independent Living. Journal of Medical Systems 2019;43:233. 
  6. Bankole A, Anderson MS, Homdee N, Alam R, Lofton A, Fyffe N, Goins H, Newbold T, Smith-Jackson T, Lach J. BESI: Behavioral and Environmental Sensing and Intervention for Dementia Caregiver Empowerment. American Journal of Alzheimer's Disease & Other Dementias 2020;35:1-15. 
  7. Miguel K, Brunete A, Hernando M. Home Camera-Based Fall Detection System for the Elderly. Sensors 2017;17(12):2864. 
  8. Serizel R, Turpault N, Shah A, Salamon J. Sound Event Detection in Synthetic Domestic Environments. IEEE International Conference on Acoustics, Speech and Signal Processing. 2020;86-90. 
  9. So SW, You SM, Kim JY, Hyun JA, Cho BH, Yook SH, Kim IY. Development of Age Classification Deep Learning Algorithm Using Korean Speech. J Biomed Eng Res. 2018;39(2):63-68 
  10. https://dcase.community/challenge2021. Accessed on 05 Feb 2023. IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events. 
  11. Abayomi OO, Damasevicius R, Qazi A, Qazi A, Adedoyin-Olowe M, Misra S. Data Augmentation and Deep Learning Methods in Sound Classification: A Systematic Review. Electronics 2022;11(22):3795. 
  12. Wei W, Zhu H, Benetos E, Wang Y. A-CRNN: A Domain Adaptation Model for Sound Event Detection. IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP. 2020; 276-280. 
  13. Kong Q, Xu Y, Wang W, Xu Y, Plumley MD. Sound Event Detection of Weakly Labelled Data With CNN-Transformer and Automatic Threshold Optimization. IEEE/ACM Transactions on Audio, Speech, and Language 2020;2450-2460. 
  14. https://firebase.google.com/?hl=ko. Accessed on 05 Feb 2023. Mobile Web Application Development Platform. 
  15. https://www.myhealthyapple.com/how-to-use-apple-watch-handwashing-feature. accessed on 01 Mar 2023. MyHealthApple.