Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Armed person detection using Deep Learning

딥러닝 기반의 무기 소지자 탐지

  • Kim, Geonuk (Department of Electronic Engineering, Kwangwoon University) ;
  • Lee, Minhun (Department of Electronic Engineering, Kwangwoon University) ;
  • Huh, Yoojin (Department of Electronic Engineering, Kwangwoon University) ;
  • Hwang, Gisu (Department of Electronic Engineering, Kwangwoon University) ;
  • Oh, Seoung-Jun (Department of Electronic Engineering, Kwangwoon University)
  • Received : 2018.09.07
  • Accepted : 2018.11.01
  • Published : 2018.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Nowadays, gun crimes occur very frequently not only in public places but in alleyways around the world. In particular, it is essential to detect a person armed by a pistol to prevent those crimes since small guns, such as pistols, are often used for those crimes. Because conventional works for armed person detection have treated an armed person as a single object in an input image, their accuracy is very low. The reason for the low accuracy comes from the fact that the gunman is treated as a single object although the pistol is a relatively much smaller object than the person. To solve this problem, we propose a novel algorithm called APDA(Armed Person Detection Algorithm). APDA detects the armed person using in a post-processing the positions of both wrists and the pistol achieved by the CNN-based human body feature detection model and the pistol detection model, respectively. We show that APDA can provide both 46.3% better recall and 14.04% better precision than SSD-MobileNet.

전 세계적으로 총기 사고는 인적이 드문 장소뿐만 아니라 사람들이 많이 모여 있는 공공장소에서도 빈번하게 일어난다. 특히, 권총과 같은 소형 총기 사고의 빈도수가 매우 높다. 그러므로 사람에 비해 상대적으로 매우 작은 크기의 객체인 권총을 가진 사람을 탐지하는 것은 사고의 피해를 최소화하는데 핵심적이다. '권총 든 사람'을 탐지하는 연구가 수행되고 있지만, 사람보다 권총은 상대적으로 크기가 작기 때문에 단일 객체만을 탐지하는 기존 객체 탐지 방법으로 '권총 든 사람'을 탐지하면 오류 발생 빈도수가 매우 높다. 이러한 문제점을 해결하기 위하여 권총으로 무장한 사람을 탐지하는 방법으로 APDA(Armed Person Detection Algorithm)를 제안한다. APDA는 입력 영상에서 합성곱신경망(Convolutional Neural Network, CNN) 기반의 인체 특징점 탐지 모델과 객체 탐지 모델을 병행하여 획득한 양 손목과 권총의 위치를 후처리 작업에서 이용하여 '권총 든 사람'을 탐지한다. APDA는 기존 방식보다 객관적 평가에서 재현율이 46.3% 향상되었고, 정밀도는 14.04% 향상되었다.

Keywords

BSGHC3_2018_v23n6_780_f0001.png 이미지

그림 1. ‘권총 든 사람’ 탐지를 위한 기존 객체 탐지 방법의 오 탐지 결과 예 Fig. 1. Examples of false detection in conventional object detection methods for detection of the armed person

BSGHC3_2018_v23n6_780_f0002.png 이미지

그림 3. APDA의 순서도 Fig. 3. A Flowchart of APDA

BSGHC3_2018_v23n6_780_f0003.png 이미지

그림 2. CMU-pose의 인체 특징점 지도 Fig. 2. A Keypoint map of CMU-pose

BSGHC3_2018_v23n6_780_f0004.png 이미지

그림 4. APDA의 두 가지 탐지 모듈 구조 (a) CMU pose, (b) SSD Fig. 4. Two major blocks for object detection in APDA: (a) CMU pose, and (b) SSD

BSGHC3_2018_v23n6_780_f0005.png 이미지

그림 5. APDA 처리 예 Fig. 5. An APDA processing example

BSGHC3_2018_v23n6_780_f0006.png 이미지

그림 6. 훈련에 사용한 두 종류 데이터 예 (a) ‘권총’ 데이터 (b) ‘손에 쥔 권총’ 데이터 Fig. 6. Examples of two kinds of pistol training data in APDA (a) ‘pistol’, and (b) ‘pistol held by hands’

BSGHC3_2018_v23n6_780_f0007.png 이미지

그림 7. APDA를 이용한 실험결과의 예 Fig. 7. An example of experimental result with APDA

BSGHC3_2018_v23n6_780_f0008.png 이미지

그림 8. APDA로 오 탐지된 예 Fig. 8. An example of false detection in APDA

표 1. ‘권총’과 ‘손에 쥔 권총’의 평균 정밀도 Table 1. Average Precision for both ‘pistol’ and ‘pistol held by hands’ data

BSGHC3_2018_v23n6_780_t0001.png 이미지

표 2. ‘손에 쥔 권총’ 학습 시 단계 별 평균 정밀도 Table 2. Average Precision per step in learning for 'pistol held by hands'

BSGHC3_2018_v23n6_780_t0002.png 이미지

표 3. SSD-MobileNet과 APDA에 대한 정밀도와 재현율 Table 3. Precision and recall values for SSD-MobileNet and APDA

BSGHC3_2018_v23n6_780_t0003.png 이미지

References

  1. Kwangsoo Kim, Ungtae Kim and Sooyeong Kwak, "Real-time Violence Video Detection based on Movement Change Characteristics" JBE, Vol.22, No. 2, pp. 234-239, March 2017, http://dx.doi.org/ 10.5909/JBE.2017.22.2.234 (accessed Aug. 1, 2018).
  2. Sanggi Kim and Dongseog Han, "Real Time Traffic Light Detection Algorithm Based on Color Map and Multilayer HOG-SVM" JBE, Vol. 22, No. 1, pp. 62-69, Jenuary 2017, http://dx.doi.org/10.5909/JBE.2017.22.1.62 (accessed Aug. 3, 2018).
  3. Seulbeen Kim and Wonjun Kim, "User Identification Method using Palm Creases and Veins based on Deep Learning" JBE, Vol. 23, No. 3, pp. 395-402, May 2018, http://dx.doi.org/10.5909/JBE.2018.23.3.395 (accessed Aug. 3, 2018).
  4. K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition" In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 770-778, 2016, https://doi.org/10.1109/cvpr.2016.90 (accessed Aug. 10, 2018).
  5. J. Hu, L. Shen and G. Sun, "Squeeze-and-Excitation Network" arXiv: 1709.01507, 2017, https://arxiv.org/pdf/1709.01507 (accessed Aug. 10, 2018).
  6. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, Cheng-Yang Fu and Alexander C. Berg, "SSD: Single Shot MultiBox Detector" In Proceeding of the European Conference on Computer Vision(ECCV), pp.21-37, 2016, https://doi.org/10.1007/978-3-319-46448-0_2 (accessed Sep 8, 2018).
  7. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto and H. Adam, "MobileNets: Efficient Convolutional Neural Network for Mobile Vision Applications" arXiv: 1704.04861, 2017, https://arxiv.org/abs/1704.04861 (accessed Sep 20, 2018).
  8. J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection" In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, 2016, https://doi.org/10.1109/cvpr.2016.91(accessed Sep 8, 2018).
  9. Z. Cao, T. Simon, Shih-E. Wei and Y. Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1302-1310, 2017, https://doi.org/10.1109/cvpr.2017.143 (accessed Sep 8, 2018).
  10. Y. Lecun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel, "Backpropagation Applied to Handwritten Zip Code Recognition" Neural Computation, vol. 1, no. 4, pp 541-551, Winter 1989, 10.1162/neco.1989.1.4.541 (accessed Aug. 5, 2018).
  11. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, June 2017, https://doi.org/10.1109/tpami.2016.2577031 (accessed Aug. 10, 2018).
  12. A. Glowacz, M. Kmiec and A. Dziech, "Towards Robust Visual Knife Detection in Images: Active Appearance Models Initialised with Shape-specific Interest Points" In Multimedia Communications, Services and Security : 5th International Conference. vol. 287, pp. 148-158, 2012, https://doi.org/10.1007/978-3-642-30721-8_15 (accessed Aug 9, 2018).
  13. L. Malagon-Borja, and O. Fuentes, "Object detection using image reconstruction with PCA" Image and Vision Computing, vol. 27, no. 1-2, pp. 2-9, 2009, https://doi.org/10.1016/j.imavis.2007.03.004 (accessed Aug 11, 2018).
  14. Derpanis KG, "The Harris corner detector" http://www.cse.yorku.ca/-kosta/CompVis_Notes/harris_detector.pdf(accessed Aug. 10, 2018)
  15. M. Grega, A. Matiolanski, P. Guzik and M. Leszczuk, "Automated Detection of Firearms and Knives in a CCTV Image" Sensors, vol. 16, no. 1. Jan 2016, https://doi.org/10.3390/s16010047 (accessed Aug 2, 2018).
  16. J. Canny, "A Computational Approach to Edge Detection" In IEEE Trans. Pattern Anal, Machine Intell., vol. PAMI-8, issue 6, pp. 679- 698, Nov 1986, https://doi.org/10.1016/b978-0-08-051581-6.50024-6 (accessed Aug 1, 2018).
  17. B.S. Manjunath, Philippe Salembier and Thomas Sikora, Introduction to MPEG-7, Multimedia Content Description Interface. Wiley, USA, 2002, https://doi.org/10.1007/springerreference_72884 (accessed Aug 15, 2018).
  18. Gyanendra K. Verma and Anamika Dhillon, "A HandHeld Gun Detection using Faster R-CNN Deep Learning" In Proceeding of the 7th International Conference on Computer and Communication Technology, pp. 84-88, November 2017, https://doi.org/10.1145/ 3154979.3154988 (accessed Aug 8, 2018).
  19. IMFDB: Internet Movie Firearms Database, http://www.imfdb.org/wiki/Main_Page (accessed Aug.15, 2018).
  20. J.M. Keller, M.R. Gray and J.A. junior, "A Fuzzy K-Nearest Neighbor Algorithm" In IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-15, issue 4, pp. 580-585, 1985, https://doi.org/10.1109/tsmc.1985.6313426 (accessed Aug 9, 2018).
  21. G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tompson, C. Bregler and K. Murphy, "Towards Accurate Multi-person Pose Estimation in the Wild" In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2017, https://doi.org/10.1109/cvpr.2017.395 (accessed Sep 5, 2018).
  22. A. Saxena, S. H. Chung and A. Y. Ng, "3-D Depth Reconstruction from a Single Still Image" International Journal of Computer Vision, Vol. 76 Issue 1, pp. 53-69, January 2008, https://doi.org/10.1007/s11263-007-0071-y (accessed Sep 1, 2018).