Browse > Article
http://dx.doi.org/10.9708/jksci.2022.27.10.001

Deep Learning-Based Companion Animal Abnormal Behavior Detection Service Using Image and Sensor Data  

Lee, JI-Hoon (School of Computer Science and Engineering, Hoseo University)
Shin, Min-Chan (School of Computer Science and Engineering, Hoseo University)
Park, Jun-Hee (School of Computer Science and Engineering, Hoseo University)
Moon, Nam-Mee (Dept. of Computer Science and Engineering, Hoseo University)
Publication Information
Journal of the Korea Society of Computer and Information / v.27, no.10, 2022 , pp. 1-9 More about this Journal
Abstract
In this paper, we propose the Deep Learning-Based Companion Animal Abnormal Behavior Detection Service, which using video and sensor data. Due to the recent increase in households with companion animals, the pet tech industry with artificial intelligence is growing in the existing food and medical-oriented companion animal market. In this study, companion animal behavior was classified and abnormal behavior was detected based on a deep learning model using various data for health management of companion animals through artificial intelligence. Video data and sensor data of companion animals are collected using CCTV and the manufactured pet wearable device, and used as input data for the model. Image data was processed by combining the YOLO(You Only Look Once) model and DeepLabCut for extracting joint coordinates to detect companion animal objects for behavior classification. Also, in order to process sensor data, GAT(Graph Attention Network), which can identify the correlation and characteristics of each sensor, was used.
Keywords
Abnormal Behavior Detection; Behavior Pattern Analysis; Multimodal Analysis; Deep Learning; Wearable Device;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 W. Kim, S. Kim, and H. Kwak, "The Companion Animal Monitoring System using Low-Power Protocol Wearable Device," Journal of the Korea Society of Computer and Information, 25, 12, pp.17-23, December 2020. DOI : 10.9708/jksci.2020.25.12.017   DOI
2 J. Lee and D. Kim, "Implementation of a Smart IoT System with Automatic Pet Feeder," Journal of Digital Contents Society, 22, 2, pp. 209-214, February 2021. DOI : 10.9728/dcs.2021.22.2.209   DOI
3 H. Kim, N. Choi, J. Kang, S. Lim and H. Kim, "Real-time Face Recognition System in CCTV Video for Crime Prevention," The Journal of Korean Institute of Information Technology,19, 8, pp.99-106, August 2021. DOI:10.14801/jkiit.2021.19.8.99   DOI
4 M. Kroschel, B. Reineking, F. Werwie, F. Wildi and I. Storch. "Remote monitoring of vigilance behavior in large herbivores using acceleration data," Animal Biotelemetry, Vol.5, No.1, pp.1-15, May 2017. DOI: 10.1186/s40317-017-0125-z   DOI
5 L. Yang, D. Jiang, X. Xia, E. Pei, C. Oveneke and H. Sahli, "Multimodal measurement of depression using deep learning models," In Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, pp. 53-59, October 2017. DOI:10.1145/3133944.3133948   DOI
6 J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You only look once: Unified, real-time object detection," In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.779-788, June 2016. DOI: 10.1109/CVPR.2016.91   DOI
7 D. Wu, S. Lv, M. Jiang and H. Song, "Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments," Computers and Electronics in Agriculture, 178, pp.1-12, August 2020. DOI:10.1016/j.compag.2020.105742   DOI
8 J. Yu, and Z. Wei, "Face mask wearing detection algorithm based on improved YOLO-v4," Sensors, 21, 9, pp.1-21, May 2021. DOI:10.3390/s21093263   DOI
9 E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka and B. Schiele, "Deepercut: A deeper, stronger, and faster multi-person pose estimation model," European Conference on Computer Vision, pp.34-50, September 2016. DOI:10.1007/978-3-319-46466-4_3   DOI
10 N. Kleanthous, A. Hussain, W. Khan, J. Sneddon and A. Mason, "Feature Extraction and Random Forest to Identify Sheep Behavior from Accelerometer Data" International Conference on Intelligent Computing. Springer, pp.408-419, October 2020. DOI:10.1007/978-3-030-60796-8_35   DOI
11 K. Sakai, K. Oishi, M. Miwa, H. Kumagai and H. Hirooka, "Behavior classification of goats using 9-axis multi sensors: The effect of imbalanced datasets on classification performance," Computers and Electronics in Agriculture, 166, pp.1-9, September 2019. DOI:10.1016/j.compag.2019.105027   DOI
12 A. Bewley, G. Zongyuan, O. Lionel, R. Fabio and U. Ben, "Simple online and realtime tracking," In 2016 IEEE international conference on image processing, pp.3464-3468, July 2016. DOI: 10.48550/arXiv.1602.00763   DOI
13 H. Xu, L. Shen, Q. Zhang and G, Cao, "Fall behavior recognition based on deep learning and image processing," International Journal of Mobile Computing and Multimedia Communications, 9, 4, pp.1-15, October 2018. DOI: 10.4018/IJMCMC.2018100101   DOI
14 R. Arablouei, L. Wang, L. Currie, F. Alvarenga, and G. Bishop-Hurley, "Animal Behavior Classification via Deep Learning on Embedded Systems," arXiv preprint arXiv:2111.12295, pp.1-13, November 2021. DOI: 10.48550/arXiv.2111.12295   DOI
15 M. Hassoon, M. Kouhi, M. Zomorodi-Moghadam and M. Abdar, "Rule optimization of boosted c5. 0 classification using genetic algorithm for liver disease prediction," In 2017 international conference on computer and applications, IEEE, pp.299-305, October 2017. DOI: 10.1109/COMAPP.2017.8079783   DOI
16 R. Vincent, A. Wagadre, A. Sivaraman, M. Rajesh and A. Rajesh, "Human Activity Recognition Using LSTM/BiLSTM," International Journal of Advanced Science and Technology, 29, 4, pp.7468-7474. April 2020.
17 S. Kang, H. Kim, Y. Ji and J. Lee, "A Study on the Acceptance Factors of the Introduction of a Smart IoT Technology for Well-being Companion Animal," The Jounal of Society for e-Business Studies, 24, 2, pp. 143-163, March 2019. DOI:10.7838/jsebs.2019.24.2.143   DOI
18 H. Shin, J. Kim, "A Study on the Development of Wearable Products Applied to PetTech Service Using IoT and AI Technology," JOURNAL OF THE KOREAN SOCIETY DESIGN CULTURE, 26, 1, pp. 261-272, March 2020. DOI : 10.18208/ksdc.2020.26.1.261   DOI
19 K. Pekka, V. Anna, S. Sanni, T. Heini, V. Heli, M. Paivi, G. Yulia, A. Christoph, S. Veikko, K. Miiamaaria, V. outi, and V. Antti, "Dog behaviour classification with movement sensors placed on the harness and the collar," Applied Animal Behaviour Science, 241, pp.1-7, June 2021. DOI: 10.1016/j.applanim.2021.105393   DOI
20 M. Salman Saeed, M. Mustafa, U. Sheikh, T. Jumani, I. Khan, S. Atawneh and N. Hamadneh, "An efficient boosted C5. 0 Decision-Tree-Based classification approach for detecting non-technical losses in power utilities," Energies, 13, 12, pp.1-19, June 2020. DOI:10.3390/en13123242   DOI
21 P. Mahto, P. Garg, P. Seth and J. Panda, "Refining Yolov4 for vehicle detection," International Journal of Advanced Research in Engineering and Technology, 11, 5, pp.409-419, July 2020.DOI:10.34218/IJARET.11.5.2020.043   DOI
22 J. Joshi, R. Goecke, S. Alghowinem, A. Dhall, M. Wagner, J. Epps, P. Gordon and M. Breakspeaer , "Multimodal assistive technologies for depression diagnosis and monitoring," Journal on Multimodal User Interfaces, 7, 3, pp.217-228, September 2013. DOI: 10.1007/s12193-013-0123-2   DOI
23 W. Chenhao, W. Yongquan, G. Dong and G, Jun, "Human behavior recognition under occlusion based on two-stream network combined with BiLSTM," In 2020 Chinese Control And Decision Conference (CCDC), pp. 3311-3316, August 2020. DOI: 10.1109/CCDC49329.2020.9164760   DOI