Browse > Article
http://dx.doi.org/10.5909/JBE.2021.26.3.283

Dense-Depth Map Estimation with LiDAR Depth Map and Optical Images based on Self-Organizing Map  

Choi, Hansol (Dept. of Computer Engineering, Kwangwoon University)
Lee, Jongseok (Dept. of Computer Engineering, Kwangwoon University)
Sim, Donggyu (Dept. of Computer Engineering, Kwangwoon University)
Publication Information
Journal of Broadcast Engineering / v.26, no.3, 2021 , pp. 283-295 More about this Journal
Abstract
This paper proposes a method for generating dense depth map using information of color images and depth map generated based on lidar based on self-organizing map. The proposed depth map upsampling method consists of an initial depth prediction step for an area that has not been acquired from LiDAR and an initial depth filtering step. In the initial depth prediction step, stereo matching is performed on two color images to predict an initial depth value. In the depth map filtering step, in order to reduce the error of the predicted initial depth value, a self-organizing map technique is performed on the predicted depth pixel by using the measured depth pixel around the predicted depth pixel. In the process of self-organization map, a weight is determined according to a difference between a distance between a predicted depth pixel and an measured depth pixel and a color value corresponding to each pixel. In this paper, we compared the proposed method with the bilateral filter and k-nearest neighbor widely used as a depth map upsampling method for performance comparison. Compared to the bilateral filter and the k-nearest neighbor, the proposed method reduced by about 6.4% and 8.6% in terms of MAE, and about 10.8% and 14.3% in terms of RMSE.
Keywords
Depth map; LiDAR; Upsampling; Self-organizing map;
Citations & Related Records
연도 인용수 순위
  • Reference
1 WG 07 MPEG 3D Graphics coding, "Text of ISO/IEC FDIS 23090-9 Geometry-based Point Cloud Compression," ISO/IEC JTC1/SC29/WG07 N00004, Online, Oct. 2020
2 C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," Sixth International Conference on Computer Vision, Bombay, India, India, 7-7 Jan. 1998.
3 T. Kohonen, "The Self-organizing Map," Proceedings of the IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 2009.   DOI
4 G. Bradski and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library, O'Reilly Media, Inc., 2008.
5 H. Hirschmuller, "Stereo Processing by Semi-Global Matching and Mutual Information," IEEE Transactions on pattern analysis and machine intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008.   DOI
6 A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, "Vision meets robotics:The kitti dataset," The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231-1237, Aug. 2013.   DOI
7 A. Hirata, R. Ishikawa, M. Roxas, and T. Oishi, "Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo," IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3806-3811, Oct. 2019.   DOI
8 T. Hui, C. Loy, and X. Tang. "Depth map super-resolution by deep multi-scale guidance," European conference on computer vision, pp. 353-369, Oct. 2016.
9 Y. Ko, H. C. Moon, H. H. Kim, J. G. Kim, "Single-Image Depth Estimation Based on CNN Using Edge Map," The Korean Society Of Broad Engineers, pp. 573-574, Jul. 2020.
10 W. Lee, J. Lee, D. Sim, S. Oh, "A Deep Learning based Inter-Layer Reference Picture Generation Method for Improving SHVC Coding Performance," pp. 401-410, JBE Vol. 24, No. 3, May. 2019.
11 K.-J. Yoon and I. S. Kweon, "Adaptive support-weight approach for correspondence search," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650 - 656, Apr. 2006.   DOI
12 N. Schneider, L. Schneider, P. Pinggera, U. Franke, M. Pollefeys, and C. Stiller, "Semantically Guided Depth Upsampling," German Conference on Pattern Recognition 2016, Hannover, Germany, 12-15 Sep. 2016.
13 Q. Yang, L. Wang, R. Yang, H. Stewe'nius, and D. Niste'r, "Stereo matching with color-weighted correlation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no.3, pp. 492 - 504, Apr. 2008.   DOI
14 L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, "High-quality real-time stereo using adaptive cost aggregation and dynamic programming," Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), Chapel Hill, NC, USA, 14-16 June 2006.
15 S. B. Imandoust and M. Bolandraftar, "Application of K-Nearest Neighbor (KNN) Approach for Predicting Economic Events: Theoretical Background," International Journal of Engineering Research and Applications, vol. 3, no. 5, pp. 605-610, Sep. 2013.
16 J. Park, J. Lee, S. Park, D. Sim, "Super Resolution Using Gradient-SR," The Korean Society Of Broad Engineers, pp. 198-199, Jul. 2018
17 J. Jung, Y. S. Ho, "Depth Map Up-sampling Using Maximum Gradient of Color Image", The Korean Society Of Broad Engineers, pp. 29-30, Nov. 2012.
18 S. Hong, Y. S. Ho, "Depth Upsampling Method Using Total Generalized Variation," JBE Vol. 21, No. 6, Nov. 2016.
19 C. Premebida, L. Garrote, A. Asvadi, A. P. Ribeiro, and U. Nunes, "High-resolution LIDAR-based Depth Mapping using Bilateral Filter," 2016 IEEE 19th international conference on intelligent transportation systems (ITSC). IEEE, Rio de Janeiro, Brazil, 1-4 Nov. 2016.
20 J. L. Schonberger and J.-M. Frahm, "Structure-from-motion revisited," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27-30 June. 2016.