Browse > Article
http://dx.doi.org/10.7746/jkros.2021.16.4.336

A Robust Deep Learning based Human Tracking Framework in Crowded Environments  

Oh, Kyungseok (School of Electronic Engineering, Kumoh National Institute of Technology)
Kim, Sunghyun (School of Electronic Engineering, Kumoh National Institute of Technology)
Kim, Jinseop (School of Electronic Engineering, Kumoh National Institute of Technology)
Lee, Seunghwan (School of Electronic Engineering, Kumoh National Institute of Technology)
Publication Information
The Journal of Korea Robotics Society / v.16, no.4, 2021 , pp. 336-344 More about this Journal
Abstract
This paper presents a robust deep learning-based human tracking framework in crowded environments. For practical human tracking applications, a target must be robustly tracked even in undetected or overcrowded situations. The proposed framework consists of two parts: robust deep learning-based human detection and tracking while recognizing the aforementioned situations. In the former part, target candidates are detected using Detectron2, which is one of the powerful deep learning tools, and their weights are computed and assigned. Subsequently, a candidate with the highest weight is extracted and is utilized to track the target human using a Kalman filter. If the bounding boxes of the extracted candidate and another candidate are overlapped, it is regarded as a crowded situation. In this situation, the center information of the extracted candidate is compensated using the state estimated prior to the crowded situation. When candidates are not detected from Detectron2, it means that the target is completely occluded and the next state of the target is estimated using the Kalman prediction step only. In two experiments, people wearing the same color clothes and having a similar height roam around the given place by overlapping one another. The average error of the proposed framework was measured and compared with one of the conventional approaches. In the error result, the proposed framework showed its robustness in the crowded environments.
Keywords
Detectron2; Human Tracking; Crowded Environment; Kalman Filter;
Citations & Related Records
연도 인용수 순위
  • Reference
1 L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaria, M. A. Fadhel, M. Al-Amidie, and L. Farhan, "Review of deep learning: concepts, CNN architectures, challenges, applications, future directions," Journal of Big Data, vol. 8, no. 53, pp. 1-74, 2021, DOI: 10.1186/s40537-021-00444-8.   DOI
2 R. Algabri and M. Choi, "Deep-Learning-Based Indoor Human Following of Mobile Robot Using Color Feature," Sensors, vol. 20, no. 9, pp. 1-19, May, 2020, DOI: 10.3390/s20092699.   DOI
3 C. Schlegel, J. Illmann, H. Jaberg, M. Schuster, and R. Worz, "Vision Based Person Tracking with a Mobile Robot," British Machine Vision Conference, Southampton, UK, pp. 418-427, Jan., 1998, DOI: 10.5244/C.12.42.   DOI
4 N. An, S. Y. Sun, X. G. Zhao, and Z. G. Hou, "Online context-based person re-identification and biometric-based action recognition for service robots," 2017 29th Chiness Control and Decision Conference, Istanbul, Turkey, pp. 3369-3374, Oct., 2017, DOI: 10.1109/CCDC.2017.7979088.   DOI
5 N. Bellotto and H. Hu, "Multisensor-Based Human Detection and Tracking for Mobile Service Robots," IEEE Transactions on Systems Man, and Cybernetics, vol. 39, pp. 167-181, Feb., 2009, DOI: 10.1109/TSMCB.2008.2004050.   DOI
6 M. T. Choi, J. S. Yeom, Y. H. Shin, and I. J. Park, "Robot-Assisted ADHD Screening in Diagnostic Process," Journal of Inteelligent and Robotic Systems, vol. 95, pp. 351-363, Jun., 2018, DOI: 10.1007/s10846-018-0890-9.   DOI
7 C. J. Seo, "A Study on Multi Target Tracking using HOG and Kalman Filter," Korean Institute of Electrical Engineers, vol. 64, no. 3, pp. 187-192, Sep., 2015, DOI: 10.5370/KIEEP.2015.64.3.187.   DOI
8 S. B. Han, S. Y. Na, S. Y. Kim, and D. H. Kim, "Real-time Human Tracking based on Distance and Color," Korean Institute of Information Scientists and Engineers, Jeju, Korea, pp. 1964-1966, 2018, [Online], https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE07503558.
9 S. Y. Park and D. H. Lee, "A Development of Visual Human Tracking System for Path Guidance Robot," Information and Control Symposium, Gunsan, Korea, pp. 39-40, 2019, [Online], https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE09262782.
10 J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger", 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 6517-6525, 2016, DOI: 10.1109/CVPR. 2017.690.   DOI
11 S. Noor, M. Waqas, M. I. Saleem, and H. N. Minhas, "Automatic Object Tracking and Segmentation Using Unsupervised SiamMask," IEEE Access, vol. 9, pp. 106550-106559, July, 2021, DOI: 10.1109/ACCESS.2021.3101054.   DOI
12 T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature Pyramid Networks for Object Detection," 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 2117-2125, 2017, DOI: 10.1109/CVPR. 2017.106.   DOI
13 E. Rublee, V. Rabaud, K. Konolige, and G. Bradski "ORB: an efficient alternative to SIFT or SURF," IEEE International Conference on Computer Vision, Barcelona, Spain, pp. 2564-2571, 2011, DOI: 10.1109/ICCV.2011.6126544.   DOI
14 K. Y. Oh, S. H. Kim, J. S. Kim, Y. Kwon, and S. H. Lee, "An Improved Deep Learning based Human Tracking Framework using Kalman Filter," KIIT Conference, Jeju, Korea, pp. 520-522, 2021, [Online], https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE10569022.