Browse > Article
http://dx.doi.org/10.9717/kmms.2020.24.2.186

Abnormal Situation Detection on Surveillance Video Using Object Detection and Action Recognition  

Kim, Jeong-Hun (Dept. of CS, Graduate School, Chungbuk National University)
Choi, Jong-Hyeok (Dept. of CS, Graduate School, Chungbuk National University)
Park, Young-Ho (Dept. of IT Eng., School of Engineering, Sookmyung Women's University)
Nasridinov, Aziz (Dept. of CS, Graduate School, Chungbuk National University)
Publication Information
Journal of Korea Multimedia Society / v.24, no.2, 2021 , pp. 186-198 More about this Journal
Abstract
Security control using surveillance cameras is established when people observe all surveillance videos directly. However, this task is labor-intensive and it is difficult to detect all abnormal situations. In this paper, we propose a deep neural network model, called AT-Net, that automatically detects abnormal situations in the surveillance video, and introduces an automatic video surveillance system developed based on this network model. In particular, AT-Net alleviates the ambiguity of existing abnormal situation detection methods by mapping features representing relationships between people and objects in surveillance video to the new tensor structure based on sparse coding. Through experiments on actual surveillance videos, AT-Net achieved an F1-score of about 89%, and improved abnormal situation detection performance by more than 25% compared to existing methods.
Keywords
Video Surveillance; Abnormal Situation Detection; Deep Neural Network;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 J.T. Zhou, J. Du, H. Zhu, X. Peng, Y. Liu, and R.S.M. Goh, "Anomalynet: An Anomaly Detection Network for Video Surveillance," IEEE Transactions on Information Forensics and Security, Vol. 14, No. 10, pp. 2538-2550, 2019.
2 J. Kang and S. Kwak, "Loitering Detection Solution for CCTV Security System," Journal of Korea Multimedia Society, Vol. 17, No. 1, pp. 15-25, 2014.   DOI
3 M. Jeon and K.J. Cheoi, "Anomaly Detection using Combination of Motion Features," Journal of Korea Multimedia Society, Vol. 21, No. 3, pp. 348-357, 2018.   DOI
4 U.J. Gim, J.H. Kim, K.H. Yoo, and A. Nasridinov, "Motion-Attentive Network for Detecting Abnormal Situations in Surveillance Video," Proceeding of ACM SIGGRAPH 2020 Posters, pp. 1-2, 2020.
5 W. Sultani, C. Chen, and M. Shah, "Real-World Anomaly Detection in Surveillance Videos," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6479-6488, 2018.
6 Y. Zhu and S. Newsam, "Motion-aware Feature for Improved Video Anomaly Detection," arXiv preprint arXiv:1907.10211, 2019.
7 U.J. Gim, J.J. Lee, J.H. Kim, Y.H. Park, and A. Nasridinov, "An Automatic Shoplifting Detection from Surveillance Videos (Student Abstract)," Proceeding of the AAAI Conference on Artificial Intelligence, pp. 13795- 13796, 2020.
8 R. Morais, V. Le, T. Tran, B. Saha, M. Mansour, and S. Venkatesh, "Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11996-12004, 2019.
9 A. Taha, H.H. Zayed, M.E. Khalifa, and E.S.M. El-Horbaty, "Human Activity Recognition for Surveillance Applications," Proceeding of the International Conference on Information Technology, pp. 577-586, 2015.
10 Z. Ren, O. Gallo, D. Sun, M.H. Yang, E. Sudderth, and J. Kautz, "A Fusion Approach for Multi-frame Optical Flow Estimation," Proceeding of IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2077-2086, 2019.
11 N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 886-893, 2005.
12 S. Yan, Y. Xiong, and D. Lin, "Spatial Temporal Graph Convolutional Network for Skeleton-based Action Recognition," Proceeding of the AAAI Conference on Artificial Intelligence (AAAI), pp. 8585-8593, 2018.
13 M. Perez, A.C. Kot, and A. Rocha, "Detection of Real-world Fights in Surveillance Videos," Proceeding of the IEEE International Conference on Acoustics, Speech and Signal P rocessing (ICASSP), pp. 2662-2666, 2019.
14 Abnormal Event CCTV Video AI Training Dataset (2019). https://aihub.or.kr/aidata/139 (accessed December 9, 2020).
15 I. Lee, D. Kim, S. Kang, and S. Lee, "Ensembl Deep Learning for Skeleton-based Action Recognition using Temporal Sliding LSTM Networks," Proceeding of the IEEE International Conference on Computer Vision, pp. 1012-1020, 2017.
16 M. Li, S. Chen, X. Chen, Y. Zhang, Y. Wang, and Q. Tian, "Actional-structural Graph Convolutional Networks for Skeleton-based Action Recognition," Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3595-3603, 2019.
17 J. Redmon and A. Farhadi, "Yolov3: An Incremental Improvement," arXiv preprint arXiv:1804.02767, pp. 1-6, 2018.
18 Z. Cao, G. Hidalgo, T. Simon, S.E. Wei, and Y. Sheikh, "OpenPose: Realtime Multi-person 2D Pose Estimation using Part Affinity Fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, No. 1, pp. 172-186, 2019.
19 D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning Spatiotemporal Features with 3D Convolutional Networks," Proceeding of the IEEE International Conference on Computer Vision (ICCV), pp. 4489-4497, 2015
20 K. Lee, I. Lee, and S. Lee, "Propagating LSTM: 3D Pose Estimation based on Joint Interdependency," Proceeding of the European Conference on Computer Vision (ECCV), pp. 119-135, 2018.