Browse > Article
http://dx.doi.org/10.9717/kmms.2022.25.6.807

Intelligent Activity Recognition based on Improved Convolutional Neural Network  

Park, Jin-Ho (Dept. of Information and Communication Engineering, Graduate School, Tongmyong University)
Lee, Eung-Joo (Department of Information & Communications Engineering, Tongmyong University)
Publication Information
Journal of Korea Multimedia Society / v.25, no.6, 2022 , pp. 807-818 More about this Journal
Abstract
In order to further improve the accuracy and time efficiency of behavior recognition in intelligent monitoring scenarios, a human behavior recognition algorithm based on YOLO combined with LSTM and CNN is proposed. Using the real-time nature of YOLO target detection, firstly, the specific behavior in the surveillance video is detected in real time, and the depth feature extraction is performed after obtaining the target size, location and other information; Then, remove noise data from irrelevant areas in the image; Finally, combined with LSTM modeling and processing time series, the final behavior discrimination is made for the behavior action sequence in the surveillance video. Experiments in the MSR and KTH datasets show that the average recognition rate of each behavior reaches 98.42% and 96.6%, and the average recognition speed reaches 210ms and 220ms. The method in this paper has a good effect on the intelligence behavior recognition.
Keywords
Action Recognition; Deep Learning; Target Detection; Convolutional Neural Network; LSTM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. Donahue, L.A. Hendricks, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-Term Recurrent Convolutional Networks for Visual Recognition and Description," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, pp. 677-691, 2016.   DOI
2 G. Yu and T. Li, "Recognition of Human Continuous Action with 3D CNN," Proceedings of the 11th International Conference on Computer Vision Systems, pp. 314-322, 2017.
3 L. Wanqing, Z. Zhengyou, and L. Zicheng, "Action Recognition Based on a Bag of 3D Points," Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 9-14, 2010.
4 G. Cheron, I. Laptev, and C. Schmid, "P-CNN: Pose-Based CNN Features for Action Recognition," Proceedings of 2015 IEEE International Conference on Computer Vision, pp. 3218-3226, 2015.
5 J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," Proceedings of 2016 IEEE Computer Vision and Pattern Recognition, pp. 779-788, 2016.
6 E.P. Ijjina and K.M. Chalavadi, "Human Action Recognition in RGB-D Videos Using Motion Sequence Information and Deep Learning," Pattern Recognition, Vol. 72, pp. 504-516, 2017.   DOI
7 L. Jun, W. Gang, D. Ling-Yu, K. Abdiyeva, and A.C. Kot, "Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks," IEEE Transactions on Image Processing, Vol. 27, No. 4, pp. 1586-1599, 2017.   DOI
8 S. Megrhi, M. Jmal, W. Souidene, and A. Beghdadi, "Spatio-Temporal Action Localization and Detection for Human Action Recognition in Big Dataset," Journal of Visual Communication and Image Representation, Vol. 41, pp. 375-390, 2016.   DOI
9 A.B. Sargano, W. Xiaofeng, P. Angelov, and Z. Habib, "Human Action Recognition Using Transfer Learning with Deep Representations," Proceedings of 2017 International Joint Conference on Neural Networks, Anchorage, pp. 463-469, 2017.
10 Z. Ning, J.-H. Park, and E.-J. Lee, "Multi-Human Behavior Recognition Based on Improved Posture Estimation Model," Journal of Korea Multimedia Society, Vol. 24, No. 5, pp. 659-666, 2021.   DOI
11 A. Ullah, J. Ahmad, K. Muhammad, M. Sajjad, and S.W. Baik, "Action Recognition in Video Sequences Using Deep Bi-directional LSTM with CNN Features," IEEE Access, Vol. 6, pp. 1155-1166, 2017.   DOI
12 A.B. Mahjoub and M. Atri, "Human Action Recognition Using RGB Data," The 11th International Design & Test Symposium, Hammamet, pp. 83-87, 2017.
13 K. Greff, R.K. Srivastava, J. Koutnik, B.R. Steunebrink, and J. Schmidhuber, "LSTM: A Search Space Odyssey," IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, No. 10, pp. 2222-2232, 2016.   DOI
14 F. Heng, "Human Behavior Recognition Based on Deep Learning," Journal of Wuhan University Information Science Edition, Vol. 41, No. 4, pp. 492-497, 2016.
15 T. Zhigang, C. Jun, L. Yikang, and L. Baoxin, "MSR-CNN: Applying Motion Salient Region Based Descriptors for Action Recognition," Proceedings of the 23rd International Conference on Pattern Recognition, pp. 3524-3529, 2016.
16 G. Stavropoulos, D. Giakoumis, K. Moustakas, and D. Tzovaras, "Automatic Action Recognition for Assistive Robots to Support MCI Patients at Home," Proceedings of the 10th International Conference on P ervasive Technologies Related to Assistive Environ- ments, pp. 366-371, 2017.
17 J.Y.H. Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici, "Beyond Short Snippets: Deep Networks for Video Classification," Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. pp. 4694-4702, 2015.