Browse > Article
http://dx.doi.org/10.7746/jkros.2022.17.3.314

Enhancing Single Thermal Image Depth Estimation via Multi-Channel Remapping for Thermal Images  

Kim, Jeongyun (Dept. of Civil and Environmental Engineering, KAIST)
Jeon, Myung-Hwan (Robotics Program, KAIST)
Kim, Ayoung (Dept. of Mechanical Engineering, SNU)
Publication Information
The Journal of Korea Robotics Society / v.17, no.3, 2022 , pp. 314-321 More about this Journal
Abstract
Depth information used in SLAM and visual odometry is essential in robotics. Depth information often obtained from sensors or learned by networks. While learning-based methods have gained popularity, they are mostly limited to RGB images. However, the limitation of RGB images occurs in visually derailed environments. Thermal cameras are in the spotlight as a way to solve these problems. Unlike RGB images, thermal images reliably perceive the environment regardless of the illumination variance but show lacking contrast and texture. This low contrast in the thermal image prohibits an algorithm from effectively learning the underlying scene details. To tackle these challenges, we propose multi-channel remapping for contrast. Our method allows a learning-based depth prediction model to have an accurate depth prediction even in low light conditions. We validate the feasibility and show that our multi-channel remapping method outperforms the existing methods both visually and quantitatively over our dataset.
Keywords
Thermal Camera; Depth Estimation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. Lu and G. Lu, "An alternative of lidar in nighttime: Unsupervised depth estimation based on single thermal image," 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2021, DOI: 10.1109/WACV48630.2021.00388.   DOI
2 C. Godard, O. M. Aodha, and G. J. Brostow, "Unsupervised monocular depth estimation with left-right consistency," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.699.   DOI
3 R. Dulski, P. Powalisz, M. Kastek, and P. Trzaskawka, "Enhancing image quality produced by IR cameras," Electro-Optical and Infrared Systems: Technology and Applications VII, 2010, DOI: 10.1117/12.864979.   DOI
4 Y. Choi, N. Kim, S. Hwang, and I. S. Kweon, "Thermal image enhancement using convolutional neural network," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea, 2016, DOI: 10.1109/IROS.2016.7759059.   DOI
5 A. N. Rajagopalan, S. Chaudhuri, and Uma Mudenagudi, "Depth estimation and image restoration using defocused stereo pairs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 11, Nov., 2004, DOI: 10.1109/TPAMI.2004.102.   DOI
6 F. J. Romero-Ramirez, R. Muoz-Salinas, and R. MedinaCarnicer, "Speeded up detection of squared fiducial markers," Image and Vision Computing, vol. 76, August, 2018, DOI: 10.1016/j.imavis.2018.05.004.   DOI
7 V. T. Tran, B.-S. Yang, F. Gu, and A. Ball, "Thermal image enhancement using bi-dimensional empirical mode decomposition in combination with relevance vector machine for rotating machinery fault diagnosis," Mechanical Systems and Signal Processing, vol. 38, no. 2, pp. 601-614, July, 2013, DOI: 10.1016/j.ymssp.2013.02.001.   DOI
8 I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, "Deeper depth prediction with fully convolutional residual networks," 2016 Fourth international conference on 3D vision (3DV), Stanford, CA, USA, 2016, DOI: 10.1109/3DV.2016.32.   DOI
9 R. Garg, B. G. V. Kumar, G. Carneiro, and I. Reid, "Unsupervised cnn for single view depth estimation: Geometry to the rescue," European Conference on Computer Vision, 2016, DOI: 10.1007/978-3-319-46484-8_45.   DOI
10 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, Apr., 2004, DOI: 10.1109/TIP.2003.819861.   DOI
11 E. Ilg, N. Mayer, T, Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, "Flownet 2.0: Evolution of optical flow estimation with deep networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, DOI: 10.1109/CVPR.2017.179.   DOI
12 S. Agaian and M. Roopaei, "Novel infrared and thermal image enhancement algorithms," Mobile Multimedia/Image Processing, Security, and Applications 2013, 2013, DOI: 10.1117/12.2016040.   DOI
13 K. Lee, J. Lee, J. Lee, S. Hwang, and S. Lee, "Brightness based convolutional neural network for thermal image enhancement," IEEE Access, vol. 5, 2017, DOI: 10.1109/ACCESS.2017.2769687.   DOI
14 S. Shah and J . K. Aggarwal, "Depth estimation u sing s tereo fish-eye lenses," 1st International Conference on Image Processing, Austin, TX, USA, 1994, DOI: 10.1109/ICIP.1994.413669.   DOI
15 N. Kim, Y. Choi, S. Hwang, and I. S. Kweon, "Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision," Thirty-Second AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018, [Online], https://ojs.aaai.org/index.php/AAAI/article/view/12297.
16 Y. Liu, X. Cao, Q. Dai, and W. Xu, "Continuous depth estimation for multi-view stereo," 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, DOI: 10.1109/CVPR.2009.5206712.   DOI
17 D. Eigen, C. Puhrsch, and R. Fergus, "Depth map prediction from a single image using a multi-scale deep network," arXiv: 1406.2283, 2014, DOI: 10.48550/arXiv.1406.2283.   DOI
18 R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, "Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 33, March, 2020, DOI: 10.1109/TPAMI.2020.3019967.   DOI
19 Z. Zhang, "A flexible new technique for camera calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, Nov., 2000, DOI: 10.1109/34.888718.   DOI
20 T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, "Unsupervised learning of depth and ego-motion from video," arXiv:1704.07813, 2017, DOI: 10.48550/arXiv.1704.07813.   DOI
21 C. Godard, O. M. Aodha, M. Firman, and G. Brostow, "Digging into self-supervised monocular depth estimation," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, DOI: 10.1109/ICCV.2019.00393.   DOI
22 Z. Li and N. Snavely, "Megadepth: Learning single-view depth prediction from internet photos," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, DOI: 10.1109/CVPR.2018.00218.   DOI
23 J. H. Lee, M.-K. Han, D. W. Ko, and I. Suh, "From big to small: Multi-scale local planar guidance for monocular depth estimation," arXiv:1907.10326, 2019, DOI: 10.48550/arXiv,1907.10326.   DOI