Browse > Article
http://dx.doi.org/10.3745/KTSDE.2021.10.5.187

LSTM(Long Short-Term Memory)-Based Abnormal Behavior Recognition Using AlphaPose  

Bae, Hyun-Jae (차세대융합기술연구원)
Jang, Gyu-Jin (차세대융합기술연구원)
Kim, Young-Hun (차세대융합기술연구원)
Kim, Jin-Pyung (차세대융합기술연구원)
Publication Information
KIPS Transactions on Software and Data Engineering / v.10, no.5, 2021 , pp. 187-194 More about this Journal
Abstract
A person's behavioral recognition is the recognition of what a person does according to joint movements. To this end, we utilize computer vision tasks that are utilized in image processing. Human behavior recognition is a safety accident response service that combines deep learning and CCTV, and can be applied within the safety management site. Existing studies are relatively lacking in behavioral recognition studies through human joint keypoint extraction by utilizing deep learning. There were also problems that were difficult to manage workers continuously and systematically at safety management sites. In this paper, to address these problems, we propose a method to recognize risk behavior using only joint keypoints and joint motion information. AlphaPose, one of the pose estimation methods, was used to extract joint keypoints in the body part. The extracted joint keypoints were sequentially entered into the Long Short-Term Memory (LSTM) model to be learned with continuous data. After checking the behavioral recognition accuracy, it was confirmed that the accuracy of the "Lying Down" behavioral recognition results was high.
Keywords
Safety Management; Action Recognition; Pose Estimation; LSTM; Deep Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Toshev, Alexander, and Christian Szegedy, "Deeppose: Human pose estimation via deep neural networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
2 Jiefeng Li, Can Wang, Hao Zhu, Yihuan Mao, Hao-Shu Fang, and Cewu Lu, "Crowdpose: Efficient crowded scenes pose estimation and a new benchmark," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
3 Xiao, Bin, Haiping Wu, and Yichen Wei, "Simple baselines for human pose estimation and tracking," Proceedings of the European Conference on Computer Vision (ECCV), 2018.
4 Yan, Sijie, Yuanjun Xiong, and Dahua Lin, "Spatial temporal graph convolutional networks for skeleton-based action recognition," Proceedings of the AAAI Conference on Artificial Intelligence, Vol.32. No.1. 2018.
5 Tsung-Yi Lin, et al., "Microsoft coco: Common objects in context," European Conference on Computer Vision, Springer, Cham, 2014.
6 Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu, "Rmpe: Regional multi-person pose estimation," Proceedings of the IEEE International Conference on Computer Vision, 2017.
7 Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang, "Deep high-resolution representation learning for human pose estimation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
8 Felix A. Gers, Jurgen Schmidhuber, and Fred Cummins, "Learning to forget: Continual prediction with LSTM," (1999): 850-855.
9 Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh, "OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields," arXiv preprint arXiv:1812.08008, 2018.
10 Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick, "Mask r-cnn," Proceedings of the IEEE International Conference on Computer Vision, 2017.
11 Zaremba, Wojciech, Ilya Sutskever, and Oriol Vinyals, "Recurrent neural network regularization," arXiv preprint arXiv:1409.2329 (2014).
12 Statistic, US Bureau of Labor, "Nonfatal Occupational Injuries and Illnesses Requiring Days Away from Work, 2011," UDo Labor, Editor (2012).
13 Human Pose Estimation Image AI Data [Internet], https://aihub.or.kr/aidata/138
14 Lieyun Ding, Weili Fang, Hanbin Luo, Peter E. D. Love, Botao Zhong, and Xi Ouyang, "A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory," Automation in Construction, Vol.86, pp.118-124, 2018.   DOI
15 D. P. Kingma, and B. Jimmy, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.