Browse > Article
http://dx.doi.org/10.4218/etrij.2018-0520

Vision-based garbage dumping action detection for real-world surveillance platform  

Yun, Kimin (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
Kwon, Yongjin (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
Oh, Sungchan (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
Moon, Jinyoung (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
Park, Jongyoul (SW.Contents Laboratory, Electronics and Telecommunications Research Institute)
Publication Information
ETRI Journal / v.41, no.4, 2019 , pp. 494-505 More about this Journal
Abstract
In this paper, we propose a new framework for detecting the unauthorized dumping of garbage in real-world surveillance camera. Although several action/behavior recognition methods have been investigated, these studies are hardly applicable to real-world scenarios because they are mainly focused on well-refined datasets. Because the dumping actions in the real-world take a variety of forms, building a new method to disclose the actions instead of exploiting previous approaches is a better strategy. We detected the dumping action by the change in relation between a person and the object being held by them. To find the person-held object of indefinite form, we used a background subtraction algorithm and human joint estimation. The person-held object was then tracked and the relation model between the joints and objects was built. Finally, the dumping action was detected through the voting-based decision module. In the experiments, we show the effectiveness of the proposed method by testing on real-world videos containing various dumping actions. In addition, the proposed framework is implemented in a real-time monitoring system through a fast online algorithm.
Keywords
action recognition; garbage dumping action; human-object relation; machine vision application; visual surveillance;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 H.W. Kuhn, The Hungarian method for the assignment problem, Naval Research Logistics Quarterly 2 (1955), 83-97.   DOI
2 P.F. Felzenszwalb, R.B. Girshick, and D.A. McAllester, Cascade object detection with deformable part models, in Proc. Comput. Vision Pattern Recogn. (CVPR), San Francisco, CA, USA, June 13-18, 2010, pp. 2241-2248.
3 J. Sklansky, Finding the convex hull of a simple polygon, Pattern Recogn. Lett. 1 (1982), 79-83.   DOI
4 K. Kang et al., Invariant‐feature based object tracking using discrete dynamic swarm optimization, ETRI J. 39 (2017), 151-162.   DOI
5 M. E. Yildirim et al., Direction‐based modified particle filter for vehicle tracking, ETRI J. 38 (2016), 356-365.   DOI
6 S. Yun et al., Action‐decision networks for visual tracking with deep reinforcement learning, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1349-1358.
7 J. F. Henriques et al., High‐speed tracking with kernelized correlation filters, IEEE Trans. Pattern Anal. Mach. Intell. 37 (2015), 583-596.   DOI
8 K. M. Yi et al., Detection of moving objects with non‐stationary cameras in 5.8ms: bringing motion detection to your mobile device, in IEEE Conf. Comput. Vision Pattern Recogn. Workshops (CVPRW), Portland, OR, USA, June 23-28, 2013, pp. 27-34.
9 C. Cortes and V. Vapnik, Support‐vector networks, Mach Learn. 20 (1995), 273-297.   DOI
10 S. Yan, Y. Xiong, and D. Lin, Spatial temporal graph convolutional networks for skeleton‐based action recognition, in Proc. AAAI Conf. Artif. Intell., New Orleans, LA, USA, Feb. 2-7, 2018.
11 C.‐B. Jin et al., Real‐time action detection in video surveillance using sub‐action descriptor with multi‐cnn, arXiv preprint, (2018), arXiv:1710.03383.
12 L. Wang et al., Temporal segment networks: Towards good practices for deep action recognition, in Proc. Eur. Conf. Comput. Vision (ECCV), Amsterdam, The Netherlands, Oct. 11-14, 2016, pp. 20-36.
13 W. Liu et al., SSD: Single shot multibox detector, in Proc. Eur. Conf. Comput. Vision (ECCV), Amsterdam, The Netherlands, Oct. 11-14, 2016, pp. 21-37.
14 J. Redmon et al., You only look once: Unified, real‐time object detection, in Proc. Comput. Vision Pattern Recogn. (CVPR), Las Vegas, NV, USA, June 27-30, 2016, pp. 779-788.
15 S. Ren et al., Faster R‐CNN: Towards real‐time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39 (2017), 1137-1149.   DOI
16 K. Simonyan and A. Zisserman, Two‐stream convolutional networks for action recognition in videos, in Proc. Adv. Nneural Inf. Process. Syst. (NIPS), Montreal, Canada, Dec. 8-13, 2014, pp. 567-576.
17 D. Tran et al., Learning spatiotemporal features with 3D convolutional networks, in Proc. Int. Conf. Comput. Vision (ICCV), Santiago, Chile, Dec. 7-13, 2015, pp. 4489-4497.
18 F. Porikli, Y. Ivanov, and T. Haga, Robust abandoned object detection using dual foregrounds, EURASIP J. Adv. Signal Process. 30 (2008), 1-11.
19 R.H. Evangelio and T. Sikora. Complementary background models for the detection of static and moving objects in crowded environments, in Proc. Adv. Video Signal‐Based Surveillance (AVSS), Klagenfurt, Austria, 2011, pp. 71-76.
20 S. Kim et al., Intelligent visual surveillance‐A survey, Int. J. Contr. Autom. Syst. 8 (2010), 926-939.   DOI
21 G. Chunhui et al., AVA: A video dataset of spatio‐temporally localized atomic visual actions, arXiv:1705.08421, 2017.
22 S. Mahankali et al., Identification of illegal garbage dumping with video analytics, in Proc. Int. Conf. Adv. Comput., Commun. Inf. (ICACCI), Bangalore, India, Sept. 19-22, 2018, pp. 2403-2407.
23 W. Kay et al., The kinetics human action video dataset, arXiv:1705.06950, 2017.
24 J. Moon et al., Extensible hierarchical method of detecting interactive actions for video understanding, ETRI J. 39 (2017), 502-513.   DOI
25 L. Cewu, J. Shi, and J. Jia. Abnormal event detection at 150 FPS in MATLAB, in Proc. Int. Conf. Comput. Vision (ICCV), Sydney, Australia, Dec. 1-8, 2013, pp. 2720-2727.
26 K. Yun, Y. Yoo, and J.Y. Choi, Motion interaction field for detection of abnormal interactions, Mach. Vis. Appl. 28 (2017), 157-171.   DOI
27 R. Csordas, L. Havasi, and T. Sziranyi, Detecting objects thrown over Fence in outdoor scenes, in Proc. Int. Conf. Comput. Vision Theory Applicat. (VISAPP), Berlin, Germany, 2015, pp. 593-599.
28 H. Begur et al., An edge‐based smart mobile service system for illegal dumping detection and monitoring in San Jose, in IEEE SmartWorld Ubiquitous Intell. Comput., Adv. Trusted Comput., Scalable Comput. Commun., Cloud Big Data Comput., Internet People Smart City Innovation, SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI, San Francisco, CA, USA, Aug. 4-8, 2017, pp. 1-6.
29 A. Dabholkar et al., Smart illegal dumping detection, in Proc. IEEE Int. Conf. Big Data Comput. Service Applicat., San Francisco, CA, USA, Apr. 6-9, 2017, pp. 255-260.
30 E. Esen, M.A. Arabaci, and M. Soysal, Fight detection in surveillance videos, in Proc. Content‐Based Multimedia Indexing (CBMI), Veszprem, Hungary, June 17-19, 2013, pp. 131-135.
31 Z. Zhang, C. Conly, and V. Athitsos, A survey on vision‐based fall detection, in Proc. ACM Int. Conf. Pervasive Technol. Related Assistive Environ., Corfu, Greece, July 1-3, 2015, pp. 1-7.
32 T. Yi Lin et al., Microsoft COCO: Common objects in context, in Proc. Eur. Conf. Comput. Vision (ECCV), Zurich, Switxerland, Sept. 6-12, 2014, pp. 740-755.
33 X. Zhou, C. Yang, and Y. Weichuan, Moving object detection by detecting contiguous outliers in the low‐rank representation, IEEE Trans. Pattern. Anal. Mach. Intell. 35 (2013), 597-610.   DOI
34 S. Javed et al., Background‐foreground modeling based on spatiotemporal sparse subspace clustering, IEEE Trans. Image Process. 26 (2017), 5840-5854.   DOI
35 B. Heo, K. Yun, and J.Y. Choi, Appearance and motion based deep learning architecture for moving object detection in moving camera, in Proc. Int. Conf. Image Process. (ICIP), Beijing, China, Sept. 17-20, 2017, pp. 1827-1831.
36 T.P. Nguyen et al., Change detection by training a triplet network for motion feature extraction, IEEE Trans. Circuits Syst. Video Technol. 29 (2019), 433-446.   DOI
37 K. Yun, J. Lim, and J.Y. Choi, Scene conditional background update for moving object detection in a moving camera, Pattern Recogn. Lett. 88 (2017), 57-63.   DOI
38 M. Andriluka et al., 2D human pose estimation: new benchmark and state of the art analysis, in Proc. Comput. Vision Pattern Recogn. (CVPR), Columbus, OH, USA, June 23-28, 2014, pp. 3686-3693.
39 Z. Cao et al., Realtime multi‐person 2D pose estimation using part affinity fields, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1302-1310.
40 G. Rogez, P. Weinzaepfel, and C. Schmid, LCR‐Net: Localization‐classification‐regression for human pose, in IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Honolulu, HI, USA, July 21-26, 2017, pp. 1216-1224.
41 H. Yoo et al., Online scheme for multiple camera multiple target tracking based on multiple hypothesis tracking, IEEE Trans. Circuits Syst. Video Technol. 27 (2017), 454-469.   DOI