Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

1D-CNN-LSTM Hybrid-Model-Based Pet Behavior Recognition through Wearable Sensor Data Augmentation

  • Hyungju Kim (Dept. of Computer Science and Engineering, Hoseo University) ;
  • Nammee Moon (Dept. of Computer Science and Engineering, Hoseo University)
  • Received : 2022.01.21
  • Accepted : 2022.08.20
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

The number of healthcare products available for pets has increased in recent times, which has prompted active research into wearable devices for pets. However, the data collected through such devices are limited by outliers and missing values owing to the anomalous and irregular characteristics of pets. Hence, we propose pet behavior recognition based on a hybrid one-dimensional convolutional neural network (CNN) and long short- term memory (LSTM) model using pet wearable devices. An Arduino-based pet wearable device was first fabricated to collect data for behavior recognition, where gyroscope and accelerometer values were collected using the device. Then, data augmentation was performed after replacing any missing values and outliers via preprocessing. At this time, the behaviors were classified into five types. To prevent bias from specific actions in the data augmentation, the number of datasets was compared and balanced, and CNN-LSTM-based deep learning was performed. The five subdivided behaviors and overall performance were then evaluated, and the overall accuracy of behavior recognition was found to be about 88.76%.

Keywords

Acknowledgement

This paper was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C2011966).

References

  1. L. Morgan, A. Protopopova, R. I. D. Birkler, B. Itin-Shwartz, G. A. Sutton, A. Gamliel, B. Yakobson, and T. Raz, "Human-dog relationships during the COVID-19 pandemic: booming dog adoption during social isolation," Humanities and Social Sciences Communications, vol. 7, article no. no. 155, 2020. https://doi.org/10.1057/s41599-020-00649-x 
  2. Z. Ng, T. C. Griffin, and L. Braun, "The new status quo: enhancing access to human-animal interactions to alleviate social isolation & loneliness in the time of COVID-19," Animals, vol. 11, no. 10, article no. 2769, 2021. https://doi.org/10.3390/ani11102769 
  3. G. Cicceri, F. De Vita, D. Bruneo, G. Merlino, and A. Puliafito, "A deep learning approach for pressure ulcer prevention using wearable computing," Human-centric Computing and Information Sciences, vol. 10, article no. 5, 2020. https://doi.org/10.1186/s13673-020-0211-8 
  4. H. Alshammari, S. A. El-Ghany, and A. Shehab, "Big IoT healthcare data analytics framework based on fog and cloud computing," Journal of Information Processing Systems, vol. 16, no. 6, pp. 1238-1249, 2020. https://doi.org/10.3745/JIPS.04.0193
  5. C. Zhu, W. Sheng, and M. Liu, "Wearable sensor-based behavioral anomaly detection in smart assisted living systems," IEEE Transactions on Automation Science and Engineering, vol. 12, no. 4, pp. 1225-1234, 2015. https://doi.org/10.1109/tase.2015.2474743 
  6. Q. Wen, L. Sun, F. Yang, X. Song, J. Gao, X. Wang, and H. Xu, "Time series data augmentation for deep learning: a survey," 2020 [Online]. Available: https://arxiv.org/abs/2002.12478. 
  7. X. Cui, V. Goel, and B. Kingsbury, "Data augmentation for deep neural network acoustic modeling," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, no. 9, pp. 1469-1477, 2015. https://doi.org/10.1109/TASLP.2015.2438544 
  8. X. Zhao, J. Sole-Casals, B. Li, Z. Huang, A. Wang, J. Cao, T. Tanaka, and Q. Zhao, "Classification of epileptic IEEG signals by CNN and data augmentation," in Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 926-930. https://doi.org/10.1109/icassp40776.2020.9052948 
  9. J. Cho and N. Moon, "Design of image generation system for DCGAN-based kids' book text," Journal of Information Processing Systems, vol. 16, no. 6, pp. 1437-1446, 2020. https://doi.org/10.3745/JIPS.02.0149 
  10. C. Shorten and T. M. Khoshgoftaar, "A survey on image data augmentation for deep learning," Journal of Big Data, vol. 6, article no. 60, 2019. https://doi.org/10.1186/s40537-019-0197-0 
  11. L. Perez and J. Wang, "The effectiveness of data augmentation in image classification using deep learning," 2017 [Online]. Available: https://arxiv.org/abs/1712.04621. 
  12. T. T. Um, F. M. Pfister, D. Pichler, S. Endo, M. Lang, S. Hirche, U. Fietzek, and D. Kulic, "Data augmentation of wearable sensor data for Parkinson's disease monitoring using convolutional neural networks," in Proceedings of the 19th ACM International Conference on Multimodal Interaction, Glasgow, UK, 2017, pp. 216-220. https://doi.org/10.1145/3136755.3136817 
  13. D. Van Der Linden, A. Zamansky, I. Hadar, B. Craggs, and A. Rashid, "Buddy's wearable is not your buddy: privacy implications of pet wearables," IEEE Security & Privacy, vol. 17, no. 3, pp. 28-39, 2019. https://doi.org/10.1109/msec.2018.2888783 
  14. H. F. Nweke, Y. W. Teh, G. Mujtaba, U. R. Alo, and M. A. Al-garadi, "Multi-sensor fusion based on multiple classifier systems for human activity identification," Human-centric Computing and Information Sciences, vol. 9, article no. 34, 2019. https://doi.org/10.1186/s13673-019-0194-5 
  15. T. Steels, B. Van Herbruggen, J. Fontaine, T. De Pessemier, D. Plets, and E. De Poorter, "Badminton activity recognition using accelerometer data," Sensors, vol. 20, no. 17, article no. 4685, 2020. https://doi.org/10.3390/s20174685 
  16. S. A. Khowaja, B. N. Yahya, and S. L. Lee, "CAPHAR: context-aware personalized human activity recognition using associative learning in smart environments," Human-centric Computing and Information Sciences, vol. 10, article no. 35, 2020. https://doi.org/10.1186/s13673-020-00240-y 
  17. A. R. Javed, M. U. Sarwar, M. O. Beg, M. Asim, T. Baker, and H. Tawfik, "A collaborative healthcare framework for shared healthcare plan with ambient intelligence," Human-centric Computing and Information Sciences, vol. 10, article no. 40, 2020. https://doi.org/10.1186/s13673-020-00245-7 
  18. Z. Xu, J. Zhao, Y. Yu, and H. Zeng, "Improved 1D-CNNs for behavior recognition using wearable sensor network," Computer Communications, vol. 151, pp. 165-171, 2020. https://doi.org/10.1016/j.comcom.2020.01.012 
  19. S. Mekruksavanich, A. Jitpattanakul, P. Youplao, and P. Yupapin, "Enhanced hand-oriented activity recognition based on smartwatch sensor data using LSTMs," Symmetry, vol. 12, no. 9, article no. 1570, 2020. https://doi.org/10.3390/sym12091570 
  20. J. Kim and N. Moon, "Dog behavior recognition based on multimodal data from a camera and wearable device," Applied Sciences, vol. 12, no. 6, article no. 3199, 2022. https://doi.org/10.3390/app12063199