The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Semantic Visual Place Recognition in Dynamic Urban Environment

동적 도시 환경에서 의미론적 시각적 장소 인식

  • Arshad, Saba (Control and Robot Engineering, Chungbuk National University) ;
  • Kim, Gon-Woo (Department of Intelligent Systems and Robotics, Chungbuk National University)
  • Received : 2022.03.10
  • Accepted : 2022.03.23
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In visual simultaneous localization and mapping (vSLAM), the correct recognition of a place benefits in relocalization and improved map accuracy. However, its performance is significantly affected by the environmental conditions such as variation in light, viewpoints, seasons, and presence of dynamic objects. This research addresses the problem of feature occlusion caused by interference of dynamic objects leading to the poor performance of visual place recognition algorithm. To overcome the aforementioned problem, this research analyzes the role of scene semantics in correct detection of a place in challenging environments and presents a semantics aided visual place recognition method. Semantics being invariant to viewpoint changes and dynamic environment can improve the overall performance of the place matching method. The proposed method is evaluated on the two benchmark datasets with dynamic environment and seasonal changes. Experimental results show the improved performance of the visual place recognition method for vSLAM.

Keywords

Acknowledgement

This research was supported in part by the MSIT (Ministry of Science and ICT), Korea, under the Grand Information Technology Research Center support program (IITP-2022-2020-0-01462) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation), and in part by the Technology Innovation Program (or Industrial Strategic Technology Development Program-ATC+) (20009546, Development of service robot core technology that can provide advanced service in real life) funded By the Ministry of Trade, Industry & Energy (MOTIE, Korea)

References

  1. M. Cummins and P. Newman, "Appearance-only SLAM at large scale with FAB-MAP 2.0," The International Journal of Robotics Research, vol. 30, no. 9, pp. 1100-1123, Aug., 2011, DOI: 10.1177/0278364910385483.
  2. E. Pepperell, P. I. Corke, and M. J. Milford, "All-environment visual place recognition with SMART," 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014, DOI: 10.1109/ICRA.2014.6907067.
  3. P. Neubert, N. Sunderhauf, and P. Protzel, "Appearance change prediction for long-term navigation across seasons," 2013 European Conference on Mobile Robots, Barcelona, Spain, 2013, DOI: 10.1109/ECMR.2013.6698842.
  4. P. Corke, R. Paul, W. Churchill, and P. Newman, "Dealing with shadows: Capturing intrinsic scene appearance for image-based outdoor localisation," 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 2013, DOI: 10.1109/IROS.2013.6696648.
  5. C. McManus, W. Churchill, W. Maddern, A. D. Stewart, and P. Newman, "Shady dealings: Robust, long-term visual localisation using illumination invariance," 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014, DOI: 10.1109/ICRA.2014.6906961.
  6. Z. Chen, O. Lam, A. Jacobson, and M. Milford, "Convolutional Neural Network-based Place Recognition," 16th Australasian Conference on Robotics and Automation 2014, 2014, [Online], https://eprints.qut.edu.au/79662.
  7. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, "NetVLAD: CNN Architecture for Weakly Supervised Place Recognition," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 40, no. 6, 2018, DOI: 10.1109/TPAMI.2017.2711011.
  8. T.-H. Wang, H.-J. Huang, J.-T. Lin, C.-W. Hu, K.-H. Zeng, and M. Sun, "Omnidirectional CNN for Visual Place Recognition and Navigation," 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 2018, DOI: 10.1109/ICRA.2018.8463173.
  9. B. Chen, D. Yuan, C. Liu, and Q. Wu, "Loop Closure Detection Based on Multi-Scale Deep Feature Fusion," Applied Sciences, vol. 9, no. 6, Mar., 2019, DOI: 10.3390/app9061120.
  10. A. Mousavian and J. Kosecka, "Semantic Image Based Geolocation Given a Map," arXiv:1609.00278, Sep., 2016, Accessed: Mar. 25, 2021, [Online], http://arxiv.org/abs/1609.00278.
  11. Y. Hou, H. Zhang, S. Zhou, and H. Zou, "Use of Roadway Scene Semantic Information and Geometry-Preserving Landmark Pairs to Improve Visual Place Recognition in Changing Environments," IEEE Access, vol. 5, pp. 7702-7713, 2017, DOI: 10.1109/ACCESS.2017.2698524.
  12. T. Naseer, G. L. Oliveira, T. Brox, and W. Burgard, "Semantics-aware visual localization under challenging perceptual conditions," 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, DOI: 10.1109/ICRA.2017.7989305.
  13. H. Xu, H. Zhang, E. Yao, and H. Song, "A Loop Closure Detection Algorithm in Dynamic Scene," International Conference on Computer, Communication and Network Technology (CCNT 2018), 2018, DOI: 10.12783/dtcse/ccnt2018/24714.
  14. S. Garg, N. Sunderhauf, M. Milford, and N. Suenderhauf, "LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints using Visual Semantics," arXiv :1804.05526, 2018, Accessed: Nov. 23, 2018, [Online], https://github.com/oravus/lostX.
  15. P. Wu, J. Wang, C. Wang, L. Zhang, and Y. Wang, "A novel fusing semantic-and appearance-based descriptors for visual loop closure detection," Optik, vol. 243, Oct., 2021, DOI: 10.1016/J.IJLEO.2021.167230.
  16. B. Chen, X. Song, H. Shen, and T. Lu, "Hierarchical Visual Place Recognition Based on Semantic-Aggregation," Applied Sciences, vol. 11, no. 20, Oct., 2021, DOI: 10.3390/APP11209540.
  17. G. Singh, M. Wu, S.-K. Lam, and D. Van Minh, "Hierarchical Loop Closure Detection for Long-term Visual SLAM with Semantic-Geometric Descriptors," 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA, 2021, DOI: 10.1109/ITSC48978.2021.9564866.
  18. Z. Yuan, K. Xu, X. Zhou, B. Deng, and Y. Ma, "SVG-Loop: Semantic-Visual-Geometric Information-Based Loop Closure Detection," Remote Sensing, vol. 13, no. 17, Sep., 2021, DOI: 10.3390/RS13173520.
  19. G. Lin, A. Milan, C. Shen, and I. Reid, "RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.549.