Browse > Article
http://dx.doi.org/10.33851/JMIS.2020.7.3.205

Detection of Dangerous Situations using Deep Learning Model with Relational Inference  

Jang, Sein (Data Fusion Research Center (DFRC))
Battulga, Lkhagvadorj (Department of Mathematics and Computer Science, Eindhoven University of Technology)
Nasridinov, Aziz (Department of Computer Science, Chungbuk National University)
Publication Information
Journal of Multimedia Information System / v.7, no.3, 2020 , pp. 205-214 More about this Journal
Abstract
Crime has become one of the major problems in modern society. Even though visual surveillances through closed-circuit television (CCTV) is extensively used for solving crime, the number of crimes has not decreased. This is because there is insufficient workforce for performing 24-hour surveillance. In addition, CCTV surveillance by humans is not efficient for detecting dangerous situations owing to accuracy issues. In this paper, we propose the autonomous detection of dangerous situations in CCTV scenes using a deep learning model with relational inference. The main feature of the proposed method is that it can simultaneously perform object detection and relational inference to determine the danger of the situations captured by CCTV. This enables us to efficiently classify dangerous situations by inferring the relationship between detected objects (i.e., distance and position). Experimental results demonstrate that the proposed method outperforms existing methods in terms of the accuracy of image classification and the false alarm rate even when object detection accuracy is low.
Keywords
Closed-circuit television; Crime; Danger detection; Smart cities;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Basharat, A. Gritai, M. Shah, "Learning object motion patterns for anomaly detection and improved object detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, pp. 1-8, 2008.
2 U.-J. Gim, J.-J. Lee, J.-H. Kim, Y.-H. Park, A. Nasridinov, "An Automatic Shoplifting Detection from Surveillance Videos," in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI2020), New York, USA, pp. 13795-13796, February 7-12, 2020.
3 M. Grega, A. Matiolanski, P. Guzik, M. Leszczuk, "Automated detection of firearms and knives in a CCTV image," Sensors, vol. 16, no. 1, pp. 1-16, 2016   DOI
4 K. He, G. Gkioxari, P. Doll'ar, R. Girshick, "Mask R-CNN," in Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2961-2969, October 22-29, 2017.
5 J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, K. Murphy, "Speed/accuracy trade-offs for modern convolutional object detectors," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 7310-7319, 2017.
6 D. Jeong, B.-G. Kim, S.-Y. Dong, "Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition," Sensors, vol. 20, no. 7, 2020.
7 S. Jang, Y.-H. Park, A. Nasridinov, "AVS-Net: Automatic Visual Surveillance Using Relation Network," in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI2019), Honolulu, USA, pp. 9947-9948, January 27-February 1, 2019.
8 J. -H. Kim, B. -G. Kim, P. P. Roy, D. Jeong, "Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure," IEEE Access, vol. 7, pp. 41273-41285, 2019.   DOI
9 Korea supreme prosecutors' office crime analysis, http://www.spo.go.kr/, 2017.
10 A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proceeding of Neural Information Processing Systems (NIPS), Nevada, USA, pp. 1106-1114, December 3-8, 2012.
11 Y. LeCun, P. Haffner, L. Bottou, Y. Bengio, "Object recognition with gradient-based learning," Shape, Contour and Grouping in Computer Vision, vol. 1681, pp. 319-345, 1999.   DOI
12 J. R. Medel, A. Savakis, "Anomaly detection in video using predictive convolutional long short-term memory networks," arXiv preprint arXiv:1612.00390.
13 M. Nair, G. Mathew, J. Neethu, J. Davies, "i-Surveillance crime monitoring and prevention using neural networks," International Research Journal of Engineering and Technology, vol. 5, no. 3, pp. 1231-1236, 2018.
14 A. Santoro, D. Raposo, D. G. Barrett, M. Malinowski, R. Pascanu, P. Battaglia, T. Lillicrap, "A simple neural network module for relational reasoning," in Proceeding of Neural Information Processing Systems, California, USA, pp. 4967-4976, Dec 4-9, 2017.
15 M. Nakib, R. T. Khan, M. S. Hasan, J. Uddin, "Crime scene prediction by detecting threatening objects using convolutional neural network," in Proceedings of the International Conference on Computer, Communication, Chemical, Material and Electronic Engineering, Rajshahi, Bangladesh, pp. 1-4, 2018.
16 S. Penmetsa, F. Minhuj, A. Singh, S. N. Omkar, "Autonomous UAV for suspicious action detection using pictorial human pose estimation and classification," Electronic Letters on Computer Vision and Image Analysis, vol. 13, no. 1, pp. 18-32, 2014.   DOI
17 J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Nevada, USA, pp. 779-788, June 26 - July 1, 2016.
18 A. Singh, D. Patil, S. N. Omkar, "Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network," arXiv preprint arXiv:1806.00746.
19 Y. Yuan, J. Chu, L. Leng, J. Miao, B.-G. Kim, "A scale-adaptive object-tracking algorithm with occlusion detection," EURASIP Journal on Image and Video Processing, vol. 7, no. 1, pp. 1-15, 2020.
20 Y. Zhu, S. Newsam, "Motion-aware feature for improved video anomaly detection," in Proceedings of British Machine Vision Conference (BMVC2019), Cardiff, UK, pp. 1-12, September 9-12, 2019.
21 M. Chebiyyam, R. D. Reddy, D. P. Dogra, H. Bhaskar, L. Mihaylova, "Motion anomaly detection and trajectory analysis in visual surveillance," Multimedia Tools and Applications, vol. 77, pp. 16223-16248, 2018.   DOI