로봇학회논문지 (The Journal of Korea Robotics Society)

  • 제17권3호
  • /
  • Pages.314-321
  • /
  • 2022
  • /
  • 1975-6291(pISSN)
  • /
  • 2287-3961(eISSN)
한국로봇학회 (Korea Robotics Society)
권호 표지
DOI QR코드

DOI QR Code

열화상 이미지 다중 채널 재매핑을 통한 단일 열화상 이미지 깊이 추정 향상

Enhancing Single Thermal Image Depth Estimation via Multi-Channel Remapping for Thermal Images

  • 투고 : 2022.03.10
  • 심사 : 2022.03.22
  • 발행 : 2022.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Depth information used in SLAM and visual odometry is essential in robotics. Depth information often obtained from sensors or learned by networks. While learning-based methods have gained popularity, they are mostly limited to RGB images. However, the limitation of RGB images occurs in visually derailed environments. Thermal cameras are in the spotlight as a way to solve these problems. Unlike RGB images, thermal images reliably perceive the environment regardless of the illumination variance but show lacking contrast and texture. This low contrast in the thermal image prohibits an algorithm from effectively learning the underlying scene details. To tackle these challenges, we propose multi-channel remapping for contrast. Our method allows a learning-based depth prediction model to have an accurate depth prediction even in low light conditions. We validate the feasibility and show that our multi-channel remapping method outperforms the existing methods both visually and quantitatively over our dataset.

키워드

과제정보

This work is supported by a grant (22TSRD-C151228-04) from Urban Declining Area Regenerative Capacity-Enhancing Technology Research Program funded by Ministry of Land, Infrastructure and Transport of Korean government

참고문헌

  1. F. J. Romero-Ramirez, R. Muoz-Salinas, and R. MedinaCarnicer, "Speeded up detection of squared fiducial markers," Image and Vision Computing, vol. 76, August, 2018, DOI: 10.1016/j.imavis.2018.05.004.
  2. E. Ilg, N. Mayer, T, Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, "Flownet 2.0: Evolution of optical flow estimation with deep networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.179.
  3. R. Dulski, P. Powalisz, M. Kastek, and P. Trzaskawka, "Enhancing image quality produced by IR cameras," Electro-Optical and Infrared Systems: Technology and Applications VII, 2010, DOI: 10.1117/12.864979.
  4. V. T. Tran, B.-S. Yang, F. Gu, and A. Ball, "Thermal image enhancement using bi-dimensional empirical mode decomposition in combination with relevance vector machine for rotating machinery fault diagnosis," Mechanical Systems and Signal Processing, vol. 38, no. 2, pp. 601-614, July, 2013, DOI: 10.1016/j.ymssp.2013.02.001.
  5. S. Agaian and M. Roopaei, "Novel infrared and thermal image enhancement algorithms," Mobile Multimedia/Image Processing, Security, and Applications 2013, 2013, DOI: 10.1117/12.2016040.
  6. Y. Choi, N. Kim, S. Hwang, and I. S. Kweon, "Thermal image enhancement using convolutional neural network," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea, 2016, DOI: 10.1109/IROS.2016.7759059.
  7. K. Lee, J. Lee, J. Lee, S. Hwang, and S. Lee, "Brightness based convolutional neural network for thermal image enhancement," IEEE Access, vol. 5, 2017, DOI: 10.1109/ACCESS.2017.2769687.
  8. S. Shah and J . K. Aggarwal, "Depth estimation u sing s tereo fish-eye lenses," 1st International Conference on Image Processing, Austin, TX, USA, 1994, DOI: 10.1109/ICIP.1994.413669.
  9. A. N. Rajagopalan, S. Chaudhuri, and Uma Mudenagudi, "Depth estimation and image restoration using defocused stereo pairs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 11, Nov., 2004, DOI: 10.1109/TPAMI.2004.102.
  10. Y. Liu, X. Cao, Q. Dai, and W. Xu, "Continuous depth estimation for multi-view stereo," 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, DOI: 10.1109/CVPR.2009.5206712.
  11. D. Eigen, C. Puhrsch, and R. Fergus, "Depth map prediction from a single image using a multi-scale deep network," arXiv: 1406.2283, 2014, DOI: 10.48550/arXiv.1406.2283.
  12. I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, "Deeper depth prediction with fully convolutional residual networks," 2016 Fourth international conference on 3D vision (3DV), Stanford, CA, USA, 2016, DOI: 10.1109/3DV.2016.32.
  13. R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, "Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 33, March, 2020, DOI: 10.1109/TPAMI.2020.3019967.
  14. N. Kim, Y. Choi, S. Hwang, and I. S. Kweon, "Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision," Thirty-Second AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018, [Online], https://ojs.aaai.org/index.php/AAAI/article/view/12297.
  15. Y. Lu and G. Lu, "An alternative of lidar in nighttime: Unsupervised depth estimation based on single thermal image," 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2021, DOI: 10.1109/WACV48630.2021.00388.
  16. Z. Zhang, "A flexible new technique for camera calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, Nov., 2000, DOI: 10.1109/34.888718.
  17. T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, "Unsupervised learning of depth and ego-motion from video," arXiv:1704.07813, 2017, DOI: 10.48550/arXiv.1704.07813.
  18. R. Garg, B. G. V. Kumar, G. Carneiro, and I. Reid, "Unsupervised cnn for single view depth estimation: Geometry to the rescue," European Conference on Computer Vision, 2016, DOI: 10.1007/978-3-319-46484-8_45.
  19. C. Godard, O. M. Aodha, M. Firman, and G. Brostow, "Digging into self-supervised monocular depth estimation," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, DOI: 10.1109/ICCV.2019.00393.
  20. C. Godard, O. M. Aodha, and G. J. Brostow, "Unsupervised monocular depth estimation with left-right consistency," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.699.
  21. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, Apr., 2004, DOI: 10.1109/TIP.2003.819861.
  22. Z. Li and N. Snavely, "Megadepth: Learning single-view depth prediction from internet photos," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, DOI: 10.1109/CVPR.2018.00218.
  23. J. H. Lee, M.-K. Han, D. W. Ko, and I. Suh, "From big to small: Multi-scale local planar guidance for monocular depth estimation," arXiv:1907.10326, 2019, DOI: 10.48550/arXiv,1907.10326.