The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Pseudo-RGB-based Place Recognition through Thermal-to-RGB Image Translation

열화상 영상의 Image Translation을 통한 Pseudo-RGB 기반 장소 인식 시스템

  • Seunghyeon Lee (Department of Intelligent Mechatronics Engineering, Sejong University) ;
  • Taejoo Kim (Department of Intelligent Mechatronics Engineering, Sejong University) ;
  • Yukyung Choi (Department of Intelligent Mechatronics Engineering, Sejong University)
  • Received : 2022.11.01
  • Accepted : 2022.11.23
  • Published : 2023.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Many studies have been conducted to ensure that Visual Place Recognition is reliable in various environments, including edge cases. However, existing approaches use visible imaging sensors, RGB cameras, which are greatly influenced by illumination changes, as is widely known. Thus, in this paper, we use an invisible imaging sensor, a long wave length infrared camera (LWIR) instead of RGB, that is shown to be more reliable in low-light and highly noisy conditions. In addition, although the camera sensor used to solve this problem is an LWIR camera, but since the thermal image is converted into RGB image the proposed method is highly compatible with existing algorithms and databases. We demonstrate that the proposed method outperforms the baseline method by about 0.19 for recall performance.

Keywords

Acknowledgement

This project was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2020R1F1A1076987 and NRF-2020M3F6A1109603)

References

  1. F. Zafari, A. Gkelias, and K. K. Leung, "A Survey of Indoor Localization Systems and Technologies," IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2568-2599, 2019, DOI: 10.1109/COMST.2019.2911558. 
  2. S. Kuutti, S. Fallah, K. Katsaros, M. Dianati, F. Mccullough, and A. Mouzakitis, "A Survey of the State-of-the-Art Localization Techniques and Their Potentials for Autonomous Vehicle Applications," IEEE Internet of Things Journal, vol. 5, no. 2, pp. 829-846, 2018, DOI: 10.1109/JIOT.2018.2812300. 
  3. X. Zhang, L. Wang, and Y. Su, "Visual place recognition: A survey from deep learning perspective." Pattern Recognition, vol. 113, May, 2021, DOI: 10.1016/j.patcog.2020.107760. 
  4. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, "NetVLAD: CNN architecture for weakly supervised place recognition," IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, DOI: 10.1109/CVPR.2016.572. 
  5. G. Berton, C. Masone, and B. Caputo, "Rethinking Visual Geo-localization for Large-Scale Applications," IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, Jun., 2022, DOI: 10.1109/CVPR52688.2022.00483. 
  6. A. Torii, R. Arandjelovic, J. Sivic, M. Okutomi, and T. Pajdla, "24/7 place recognition by view synthesis," IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, DOI: 10.1109/CVPR.2015.7298790. 
  7. J. Hyeon, J. Kim, and N. Doh, "Pose correction for highly accurate visual localization in large-scale indoor spaces," IEEE/CVF International Conference on Computer Vision, Montreal, Canada, Oct., 2021, DOI: 10.1109/ICCV48922.2021.01567. 
  8. E. Ershov, A. Savchik, D. Shepelev, N. Banic, M. S. Brown, R. Timofte, K. Koscevic, M. Freeman, V. Tesalin, D. Bocharov, I. Semenkov, M. Subasic, S. Loncaric, A. Terekhin, S. Liu, C. Feng, H. Wang, R. Zhu, Y. Li, L. Lei, Z. Li, S. Yi, L. Han, R. Wu, X. Jin, C. Guo, F. Kinli, S. Mentes, B. Ozcan, F. Kirac, S. Zini, C. Rota, M. Buzzelli, S. Bianco, R. Schettini, W. Li, Y. Ma, T. Wang, R. Xu, F. Song, W. Chen, H. Yang, Z. Huang, H. Chang, S. Kuo, Z. Liang, S. Zhou, R. Feng, C. Li, X. Chen, B. Song, S. Zhang, L. Liu, Z. Wang, D. Ryu, H. Bae, T. Kwon, C. Desai, N. Akalwadi, A. Joshi, C. Mandi, S. Malagi, A. Uppin, S. S. Reddy, R. A. Tabib, U. Patil, and U. Mudenagudi, "NTIRE 2022 challenge on night photography rendering," IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, Jun., 2022, DOI: 10.1109/CVPRW56347.2022.00135. 
  9. T. Naseer, W. Burgard, and C. Stachniss, "Robust Visual Localization Across Seasons," IEEE Transactions on Robotics, vol. 34, no. 2, pp. 289-302, Apr., 2018, DOI: 10.1109/TRO.2017.2788045. 
  10. A. Anoosheh, T. Sattler, R. Timofte, M. Pollefeys, and L. V. Gool, "Night-to-Day Image Translation for Retrieval-based Localization," International Conference on Robotics and Automation, Montreal, Canada, May., 2019, DOI: 10.1109/ICRA.2019.8794387. 
  11. V. Tsagaris and V. Anastassopoulos, "Fusion of visible and infrared imagery for night color vision," Displays, vol. 26, No. 4-5, pp. 191-196, Oct., 2005, DOI: 10.1016/j.displa.2005.06.007. 
  12. D. Han, Y. Hwang, N. Kim, and Y. Choi, "Multispectral Domain Invariant Image for Retrieval-based Place Recognition," IEEE International Conference on Robotics and Automation, Paris, France, 2020, DOI: 10.1109/ICRA40945.2020.9197514. 
  13. Y. Yuan, R. Fu, L. Huang, W. Lin, C. Zhang, X. Chen, and J. Wang, "Hrformer: High-resolution vision transformer for dense predict," Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021, [Online], https://proceedings.neurips.cc/paper/2021/hash/3bbfdde8842a5c44a0323518eec97cbe-Abstract.html. 
  14. Y. Choi, N. Kim, S. Hwang, K. Park, J. Yoon, K. An, and I. Kweon, "KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 3, pp. 934-948, Mar., 2018, DOI: 10.1109/TITS.2018.2791533. 
  15. D. Hendrycks and T. Dietterich, "Benchmarking neural network robustness to common corruptions and perturbations," Machine Learning, 2019, DOI: 10.48550/arXiv.1903.12261.