DOI QR코드

DOI QR Code

Real-Time Correction Based on wheel Odometry to Improve Pedestrian Tracking Performance in Small Mobile Robot

소형 이동 로봇의 사람 추적 성능 개선을 위한 휠 오도메트리 기반 실시간 보정에 관한 연구

  • Received : 2022.02.25
  • Accepted : 2022.03.22
  • Published : 2022.05.31

Abstract

With growth in intelligence of mobile robots, interaction with humans is emerging as a very important issue for mobile robots and the pedestrian tracking technique following the designated person is adopted in many cases in a way that interacts with humans. Among the existing multi-object tracking techniques for pedestrian tracking, Simple Online and Realtime Tracking (SORT) is suitable for small mobile robots that require real-time processing while having limited computational performance. However, SORT fails to reflect changes in object detection values caused by the movement of the mobile robot, resulting in poor tracking performance. In order to solve this performance degradation, this paper proposes a more stable pedestrian tracking algorithm by correcting object tracking errors caused by robot movement in real time using wheel odometry information of a mobile robot and dynamically managing the survival period of the tracker that tracks the object. In addition, the experimental results show that the proposed methodology using data collected from actual mobile robots maintains real-time and has improved tracking accuracy with resistance to the movement of the mobile robot.

Keywords

Acknowledgement

This research was supported by the MOTIE (Ministry of Trade, Industry, and Energy) in Korea, under Global Talent Growth Support Project for Robot Industry Innovation (P0017311) supervised by the Korea Institute for Advancement of Technology (KIAT)

References

  1. R. Bogue, "Growth in e-commerce boosts innovation in the warehouse robot market," Industrial Robot, 2016, DOI: 10.1108/IR-07-2016-0194.
  2. N. Hitti. Ballie the rolling robot is samsung's near-future vision of personal care, 2020, [Online], https://www.dezeen.com/2020/01/08/samsung-ballie-robot-ces-2020/.
  3. L. Leal-Taixe, A. Milan, I. Reid, S. Roth, and K. Schindler, Motchallenge 2015: Towards a benchmark for multi-target tracking," arXiv preprint arXiv:1504.01942, 2015, DOI: 10.48550/arXiv.1504.01942.
  4. P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taix e, "Mot20: A benchmark for multi object tracking in crowded scenes," arXiv: 2003.09003, 2020, [Online], DOI: 10.48550/arXiv.2003.09003.
  5. Y. Xu, Y. Ban, G. Delorme, C. Gan, D. Rus, and X. Alameda-Pineda, "Transcenter: Trans-formers with dense queries for multiple-object tracking," arXiv:2103.15145, 2021, DOI: 10.48550/arXiv.2103.15145.
  6. D. Stadler and J. Beyerer, "Improving multiple pedes- trian tracking by track management and occlusion handling," IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, DOI: 10.1109/cvpr46437.2021.01081.
  7. A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, "Simple online and realtime tracking," 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 2016, DOI: 10.1109/icip.2016.7533003.
  8. S. Kapania, D. Saini, S. Goyal, N. Thakur, R. Jain, and P. Nagrath, "Multi object tracking with uavs using deep sort and yolov3 retinanet detection frame- work," 1st ACM Workshop on Autonomous and Intelligent Mobile Systems, 2020, DOI: 10.1145/3377283.3377284.
  9. E. Bochinski, V. Eiselein, and T. Sikora, "High-Speed tracking-by-detection without using image information," 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, DOI: 10.1109/avss.2017.8078516.
  10. R. Pereira, G. Carvalho, L. Garrote, and U. J. Nunes, "Sort and Deep-SORT Based Multi-Object Tracking for Mobile Robotics: Evaluation with New Data Association Metrics," Applied Sciences, vol. 12, no. 3, 2022, DOI: 10.3390/app12031319.
  11. T. Horiuchi, S. Thompson, S. Kagami, and Y. Ehara, "Pedestrian tracking from a mobile robot using a laser range finder," 2007 IEEE International Conference on Systems, Man and Cybernetics, Montreal, QC, Canada, 2007, DOI: 10.1109/icsmc.2007.4413964.
  12. D. Reid, "An algorithm for tracking multiple targets," IEEE Transactions on Automatic Control, vol. 24, no. 6, Dec., 1979, DOI: 10.1109/tac.1979.1102177.
  13. N. A Tsokas and K. J. Kyriakopoulos, "Multi-robot multiple hypothesis tracking for pedestrian tracking," Autonomous Robots, vol. 32, no.1, 2012, DOI: 10.1007/s10514-011-9259-7.
  14. B. Nam, S.-I. Kang, and H. Hong, "Pedestrian detection system based on stereo vision for mobile robot," 2011 17th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), Ulsan, South Korea, 2011, DOI: 10.1109/fcv.2011.5739758.
  15. F. Basso, M. Munaro, S. Michieletto, E. Pagello, and E. Menegatti, "Fast and Robust Multi-people Tracking from RGB-D Data for a Mobile Robot," Intelligent Autonomous Systems 12, 2013, DOI: 10.1007/978-3-642-33926-4_25.
  16. H. Zhang, C. Reardon, and L. E. Parker, "Real-Time Multiple Human Perception With Color-Depth Cameras on a Mobile Robot," IEEE Transactions on Cybernetics, vol. 43, no. 5, Oct., 2013, DOI: 10.1109/tcyb.2013.2275291.
  17. Z. Sun, J. Chen, L. Chao, W. Ruan, and M. Mukherjee, "A Survey of Multiple Pedestrian Tracking Based on Tracking-by-Detection Framework," IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 5, May, 2020, DOI: 10.1109/tcsvt.2020.3009717.
  18. S. Hossain and D.-J. Lee, "Deep learning-based real- time multiple-object detection and tracking from aerial imagery via a flying robot with gpu-based embedded devices," Sensors, vol. 19, no. 15, 2019, DOI: 10.3390/s19153371.
  19. L. A. Nguyen, P. T. Dung, T. D. Khoa, N. H. Son, N. T. Hiep, P. V. Nguyen, V. D. Truong, D. H. Toan, N. M. Hung, T.- D. Ngo, and X.-T. Truong, "Deep learning-based multiple objects detection and tracking system for socially aware mobile robot navigation framework," 2018 5th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh City, Vietnam, 2018, DOI: 10.1109/nics.2018.8606878.
  20. G. de S. Carvalho, "Kalman Filter-Based Object Tracking Techniques for Indoor Robotic Applications," PhD thesis, University of Coimbra, Coimbra, Portugal, 2021, [Online], https://estudogeral.sib.uc.pt/handle/10316/98163.
  21. G. Koch, "Siamese neural networks for one-shot image recognition," M.S thesis, University of Toronto, Toronto, Canada, 2015, [Online], http://www.cs.toronto.edu/~gkoch/files/msc-thesis.pdf.
  22. D. S Bolme, J. R. Beveridge, B. A. Draper, and Y. M, Lui, "Visual object tracking using adaptive correlation filters," 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010, DOI: 10.1109/cvpr.2010.5539960.
  23. N. Wojke, A. Bewley, and D. Paulus, "Simple online and realtime tracking with a deep association metric," 2017 IEEE international conference on image processing (ICIP), Beijing, China, 2017, DOI: 10.1109/icip.2017.8296962.
  24. D. Scaramuzza and F. Fraundorfer, "Visual odometry [tutorial]," IEEE Robotics & Automation Magazine, vol. 18, no. 4, Dec., 2011, DOI: 10.1109/mra.2011.943233.
  25. Z. Z. Nejad and A. H. Ahmadabadian, "ARM-VO: an efficient monocular visual odometry for ground vehicles on ARM CPUs," Machine Vision and Applications, vol. 30, no. 6, 2019, DOI: 10.1007/s00138-019-01037-5.
  26. E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, "Performance Measures and a Data Set for Multi-target, Multi-camera Tracking," European Conference on Computer Vision, 2016, DOI: 10.1007/978-3-319-48881-3_2.
  27. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, DOI: 10.1109/cvpr.2016.91.
  28. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C Berg, "SSD: Single Shot MultiBox Detector," European Conference on Computer Vision, 2016, DOI: 10.1007/978-3-319-46448-0_2.
  29. A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv: 2004.10934, 2020, DOI: 10.48550/arXiv.2004.10934.
  30. C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, "Scaled-YOLOv4: Scaling Cross Stage Partial Network," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, DOI: 10.1109/cvpr46437. 2021.01283.
  31. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, DOI: 10.1109/cvpr.2018.00474.
  32. A. Milan, L. Leal-Taixe, I. Reid, S. Roth, and K. Schindler, "MOT16: A Benchmark for Multi-Object Tracking," arXiv: 1603.00831, 2016, DOI: 10.48550/arXiv.1603.00831.