The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

A Deep Convolutional Neural Network Based 6-DOF Relocalization with Sensor Fusion System

센서 융합 시스템을 이용한 심층 컨벌루션 신경망 기반 6자유도 위치 재인식

  • Jo, HyungGi (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Cho, Hae Min (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Lee, Seongwon (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Kim, Euntai (Dept. of Electrical and Electronic Engineering, Yonsei University)
  • Received : 2018.12.08
  • Accepted : 2019.02.06
  • Published : 2019.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a 6-DOF relocalization using a 3D laser scanner and a monocular camera. A relocalization problem in robotics is to estimate pose of sensor when a robot revisits the area. A deep convolutional neural network (CNN) is designed to regress 6-DOF sensor pose and trained using both RGB image and 3D point cloud information in end-to-end manner. We generate the new input that consists of RGB and range information. After training step, the relocalization system results in the pose of the sensor corresponding to each input when a new input is received. However, most of cases, mobile robot navigation system has successive sensor measurements. In order to improve the localization performance, the output of CNN is used for measurements of the particle filter that smooth the trajectory. We evaluate our relocalization method on real world datasets using a mobile robot platform.

Keywords

References

  1. R. Li, Q. Liu, J. Gui, D. Gu, and H. Hu, "Indoor Relocalization in Challenging Environments With Dual-Stream Convolutional Neural Networks," IEEE Transaction on Automation Science and Engineering, vol. 15, no. 2, pp. 651-662, Apr. 2018. https://doi.org/10.1109/TASE.2017.2664920
  2. J. L. Schonberger and J.-M. Frahm, "Structure-from-Motion Revisited," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104-4113, Jun. 2016.
  3. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), New York, NY, USA, 2006, DOI: 10.1109/CVPR.2006.264.
  4. A. Kendall, M. Grimes, and R. Cipolla, "PoseNet: A convolutional network for real-time 6-DOF camera relocalization," 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 2938-2946, 2015.
  5. A. Kendall and R. Cipolla, "Modelling uncertainty in deep learning for camera relocalization," 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, pp. 4762-4769, 2016.
  6. S. Thrun, D. Fox, W. Burgard, and F. Dellaert, "Robust Monte Carlo localization for mobile robots," Artificial Intelligence, vol. 128, no. 1-2, pp. 99-141, May. 2001. https://doi.org/10.1016/S0004-3702(01)00069-8
  7. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, "NetVLAD: CNN Architecture for Weakly Supervised Place Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 6, pp. 1437-1451, Jun., 2018. https://doi.org/10.1109/TPAMI.2017.2711011
  8. J. Shotton, B. Glocker, C. Zach, S. Izadi, A. Criminisi, and A. Fitzgibbon, "Scene coordinate regression forests for cam- era relocalization in RGB-D images," 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, pp. 2930-2937, 2013.
  9. E. Brachmann, A. Krull, S. Nowozin, J. Shotton, F. Michel, S. Gumhold, and C. Rother, "DSAC - Differentiable RANSAC for Camera Localization," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.267.
  10. Z. Chen, O. Lam, A. Jacobson, and M. Milford, "Convolutional neural network-based place recognition," arXiv:1411.1509 [cs.CV], 2014.
  11. R. Clark, S. Wang, A. Markham, N. Trigoni, and H. Wen, "VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 6856-6864, 2017.
  12. F. Walch, C. Hazirbas, L. Leal-Taixe, T. Sattler, S. Hilsenbeck, and D. Cremers, "Image-based localization using LSTMs for structured feature correlation," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 627-637, 2017.
  13. M. Velas, M. Spanel, Z. Materna, and A. Herout, "Calibration of RGB Camera With Velodyne LiDAR." WSCG 2014, pp. 135-144. 2014.
  14. H. Jo, H. M. Cho, S. Lee, and E. Kim, "Multi-Resolution Point Cloud Generation Based on Heterogeneous Sensor Fusion System," 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, South Korea, pp. 886-888, 2017.
  15. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, DOI: 10.1109/CVPR.2015.7298594.
  16. H. Jo, H. M. Cho, S. Jo, and E. Kim, "Efficient Grid-Based Rao-Blackwellized Particle Filter SLAM With Interparticle Map Sharing," IEEE Transactions on Mechatronics, vol. 23, no. 2, pp. 714-724, Apr. 2018. https://doi.org/10.1109/TMECH.2018.2795252

Cited by

  1. 지구 물리정보를 이용한 무인잠수정의 복합 항법 기술 vol.15, pp.1, 2020, https://doi.org/10.7746/jkros.2020.15.1.024