Browse > Article
http://dx.doi.org/10.3745/JIPS.02.0154

RAVIP: Real-Time AI Vision Platform for Heterogeneous Multi-Channel Video Stream  

Lee, Jeonghun (Dept. of Embedded Systems Engineering, Incheon National University)
Hwang, Kwang-il (Dept. of Embedded Systems Engineering, Incheon National University)
Publication Information
Journal of Information Processing Systems / v.17, no.2, 2021 , pp. 227-241 More about this Journal
Abstract
Object detection techniques based on deep learning such as YOLO have high detection performance and precision in a single channel video stream. In order to expand to multiple channel object detection in real-time, however, high-performance hardware is required. In this paper, we propose a novel back-end server framework, a real-time AI vision platform (RAVIP), which can extend the object detection function from single channel to simultaneous multi-channels, which can work well even in low-end server hardware. RAVIP assembles appropriate component modules from the RODEM (real-time object detection module) Base to create per-channel instances for each channel, enabling efficient parallelization of object detection instances on limited hardware resources through continuous monitoring with respect to resource utilization. Through practical experiments, RAVIP shows that it is possible to optimize CPU, GPU, and memory utilization while performing object detection service in a multi-channel situation. In addition, it has been proven that RAVIP can provide object detection services with 25 FPS for all 16 channels at the same time.
Keywords
Multi-Channel; Multi-Stream; Object Detection; Surveillance Systems; Vision;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Wikipedia, "Digital video recorder," [Online]. Available: https://en.wikipedia.org/wiki/Digital_video_recorder.
2 Wikipedia, "Network video recorder," [Online]. Available: https://en.wikipedia.org/wiki/Network_video_recorder.
3 N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, 2005, pp. 886-893.
4 D. G. Lowe, "Object recognition from local scale-invariant features," in Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, 1999, pp. 1150-1157.
5 C. P. Papageorgiou, M. Oren, and T. Poggio, "A general framework for object detection," in Proceedings of the 6th International Conference on Computer Vision (IEEE Cat. No. 98CH36271), Bombay, India, 1998, pp. 555-562.
6 J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: unified, real-time object detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 2016, pp. 779-788.
7 J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 7263-7271.
8 Y. Wu, Z. Zhao, S. Zhang, L. Yao, Y. Yang, T. Z. Fu, and S. Winkler, "Interactive multi-camera soccer video analysis system," in Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019, pp. 1047-1049.
9 G. Mattela, C. Pal, M. Tripathi, R. Gavval, and A. Acharyya, "Enterprise class deep neural network architecture for recognizing objects and faces for surveillance systems," in Proceedings of 2019 11th International Conference on Communication Systems & Networks (COMSNETS), Bengaluru, India, 2019, pp. 607-612. IEEE.
10 J. S. Park, M. Wiranegara, and G. Y. Son, "Multi-channel video analysis based on deep learning for video surveillance," The Journal of the Korea Institute of Electronic Communication Sciences, vol. 13, no. 6, pp. 1263-1268, 2018.   DOI
11 S. Aslan and S. C. Ileri, "Performance analysis of ARM big.LITTLE architecture based mobile processor with multi-thread face detection," in Proceedings of 2019 4th International Conference on Computer Science and Engineering (UBMK), Samsun, Turkey, 2019, pp. 336-339.
12 X. Wang, "An efficient end-to-end object detection pipeline on GPU using CUDA," Master's thesis, Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, The Netherlands, 2019.
13 S. Liu, G. Liu, and H. Zhou, "A robust parallel object tracking method for illumination variations," Mobile Networks and Applications, vol. 24, no. 1, pp. 5-17, 2019.   DOI
14 H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a transport protocol for real-time applications," Internet Engineering Task Force, Fremont, CA, RFC 3550, Standard 64, 2003.
15 J. Redmon and A. Farhadi, "Yolov3: an incremental improvement," 2018 [Online]. Available: https://arxiv.org/abs/1804.02767.
16 S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.   DOI
17 J. Parsola, D. Gangodkar, and A. Mittal, "Post event investigation of multi-stream video data utilizing Hadoop cluster," International Journal of Electrical & Computer Engineering, vol. 8, no. 6, pp. 5089-5097, 2018.   DOI
18 S. Karimi-Mansoub, R. Abri, and A. Yarici, "Concurrent real-time object detection on multiple live streams using optimization CPU and GPU resources in YOLOv3," in Proceedings of the 4th International Conference on Advances in Signal, Image and Video Processing, Athens, Greece, 2019, pp. 23-28.
19 L. Tan, X. Dong, Y. Ma, and C. Yu, "A multiple object tracking algorithm based on YOLO detection," in Proceedings of 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Beijing, China, 2018, pp. 1-5.
20 H. Schulzrinne, A. Rao, and R. Lanphier, "Real Time Streaming Protocol (RTSP)," Internet Engineering Task Force, Fremont, CA, RFC 2326, 1998.
21 D. Kim and S. Park, "An intelligent collaboration framework between edge camera and video analysis system," in Proceedings of 2018 International Conference on Electronics, Information, and Communication (ICEIC), Honolulu, HI, 2018, pp. 1-3.
22 NVIDIA, "DeepStream SDK 5.1," 2021 [Online]. Available: https://developer.nvidia.com/deepstream-getting-started.
23 R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 580-587.