Browse > Article
http://dx.doi.org/10.7472/jksii.2021.22.3.17

Intelligent Abnormal Situation Event Detections for Smart Home Users Using Lidar, Vision, and Audio Sensors  

Kim, Da-hyeon (Dept. of Software, Korea National University of Transportation)
Ahn, Jun-ho (Dept. of Software, Korea National University of Transportation)
Publication Information
Journal of Internet Computing and Services / v.22, no.3, 2021 , pp. 17-26 More about this Journal
Abstract
Recently, COVID-19 has spread and time to stay at home has been increasing in accordance with quarantine guidelines of the government such as recommendations to refrain from going out. As a result, the number of single-person households staying at home is also increasingsingle-person households are less likely to be notified to the outside world in times of emergency than multi-person households. This study collects various situations occurring in the home with lidar, image, and voice sensors and analyzes the data according to the sensors through their respective algorithms. Using this method, we analyzed abnormal patterns such as emergency situations and conducted research to detect abnormal signs in humans. Artificial intelligence algorithms that detect abnormalities in people by each sensor were studied and the accuracy of anomaly detection was measured according to the sensor. Furthermore, this work proposes a fusion method that complements the pros and cons between sensors by experimenting with the detectability of sensors for various situations.
Keywords
Smart home; Intelligent; Abnormal detections; Sensor fusing; Algorithm;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Korea Mutual-aid News, "The soaring rise of elderly lonely deaths, Urgent need for a social safety net", 2020. https://www.kongje.or.kr/news/articleView.html?idxno=880
2 Csaba Benedek, "3D people surveillance on range data sequences of a rotating Lidar", Pattern Recognition Letters, Vol. 50, pp. 149-158, 2014. https://doi.org/10.1016/j.patrec.2014.04.010   DOI
3 C. Premebida, O. Ludwig and U. Nunes, "Exploiting LIDAR-based features on pedestrian detection in urban scenarios", 12th International IEEE Conference on Intelligent Transportation Systems, pp. 1-6, 2009. https://doi.org/10.1109/ITSC.2009.5309697   DOI
4 Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", arXiv, 2014. https://arxiv.org/abs/1311.2524
5 Ross Girshick, "Fast R-CNN", International Conference on Computer Vision(ICCV), 2015. https://arxiv.org/abs/1504.08083
6 He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian, "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", Lecture Notes in Computer Science, pp. 346-361, 2014. http://dx.doi.org/10.1007/978-3-319-10578-9_23   DOI
7 Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", arXiv, 2016. https://arxiv.org/abs/1506.02640
8 L. Vuegen, B. V. Broeck, P. Karsmakers, J. Gemmeke, B. Vanrumste, H. V. hamme, "An MFCC GMM approach for event detection and classification", IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events, 2013. https://limo.libis.be/primo-explore/fulldisplay?docid=LIRIAS2349843&context=L&vid=Lirias&search_scope=Lirias&tab=default_tab&lang=en_US
9 Xilin Yu, Laishuan Wang, Xian Zhao, Chunmei Lu, Xi Long, Wei Chen, "An Investigation into Audio Features and DTW Algorithms for Infant Cry Classification", Association for Computing Machinery, No. 06, pp. 54-59, 2019. https://doi.org/10.1145/3375923.3375929   DOI
10 S. Cosar, G. Donatiello, V. Bogorny, C. Garate, L. O. Alvares and F. Bremond, "Toward Abnormal Trajectory and Event Detection in Video Surveillance", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, No. 3, pp. 683-695, March 2017. https://doi.org/10.1109/TCSVT.2016.2589859   DOI
11 Huy Phan, Lars Hertel, Marco Maass, Philipp Koch, Radoslaw Mazur, Alfred Mertins, "Improved Audio Scene Classification Based on Label-Tree Embeddings and Convolutional Neural Networks", IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 25, 2017. https://doi.org/10.1109/TASLP.2017.2690564   DOI
12 The Science Times, "How has your lifestyle changed since COVID-19?", 2020. https://www.sciencetimes.co.kr/news/%EC%BD%94%EB%A1%9C%EB%82%98-%EC%9D%B4%ED%9B%84-%EB%9D%BC%EC%9D%B4%ED%94%84-%EC%8A%A4%ED%83%80%EC%9D%BC%EC%9D%80-%EC%96%B4%EB%96%BB%EA%B2%8C-%EB%B3%80%ED%96%88%EB%82%98/#.YCvUs4HVA8Y.link
13 Yonhapnews, "AI speaker saved for the elderly living alone", 2020. https://www.yna.co.kr/view/AKR20200803102200052
14 Z. Ma, J. Bigham, S. Poslad, B. Wu, X. Zhang and E. Bodanese, "Device-Free, Activity During Daily Life, Recognition Using a Low-Cost Lidar", IEEE Global Communications Conference (GLOBECOM) ,pp. 1-6, 2018. https://doi.org/10.1109/GLOCOM.2018.8647251   DOI
15 The Statistics Korea, "2020 Statistics show single-person households", 2020. http://kostat.go.kr/portal/korea/kor_nw/1/1/index.board?bmode=read&aSeq=386517
16 M. Przybyla, "Detection and tracking of 2D geometric obstacles from LRF data," 2017 11th International Workshop on Robot Motion and Control (RoMoCo), pp. 135-141, 2017. https://doi.org/10.1109/RoMoCo.2017.8003904   DOI
17 Tslil, Or and Elbaz, Amit and Feiner, Tal and Carmi, Avishy, "Representing and updating objects' identities in semantic SLAM", IEEE 23rd International Conference on Information Fusion (FUSION), Rustenburg, South Africa, pp. 1-7, 2020. https://doi.org/10.23919/FUSION45008.2020.9190524   DOI
18 Ju-Ho Jung, Jun-Ho Ahn, "Intelligent User Pattern Recognition based on Vision, Audio and Activity for Abnormal Event Detections of Single Households", Korean Society of Computer Information, Vol. 24, No. 5, pp. 59-66, 2019. https://doi.org/10.9708/JKSCI.2019.24.05.059   DOI
19 R. Bardeli and D. Stein, "Uninformed Abnormal Event Detection on Audio," Speech Communication, 10. ITG Symposium, Braunschweig, Germany, pp. 1-4, 2012. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6309608
20 Dan Jia, Mats Steinweg, Alexander Hermans, Bastian Leibe, "Self-Supervised Person Detection in 2D Range Data using a Calibrated Camera", arXiv, 2020. https://arxiv.org/abs/2012.08890