Browse > Article
http://dx.doi.org/10.12815/kits.2021.20.6.313

Efficient Self-supervised Learning Techniques for Lightweight Depth Completion  

Park, Jae-Hyuck (Autonomous Driving Intelligence Research Section, ETRI)
Min, Kyoung-Wook (Autonomous Driving Intelligence Research Section, ETRI)
Choi, Jeong Dan (Intelligent Robotics Research Division, ETRI)
Publication Information
The Journal of The Korea Institute of Intelligent Transport Systems / v.20, no.6, 2021 , pp. 313-330 More about this Journal
Abstract
In an autonomous driving system equipped with a camera and lidar, depth completion techniques enable dense depth estimation. In particular, using self-supervised learning it is possible to train the depth completion network even without ground truth. In actual autonomous driving, such depth completion should have very short latency as it is the input of other algorithms. So, rather than complicate the network structure to increase the accuracy like previous studies, this paper focuses on network latency. We design a U-Net type network with RegNet encoders optimized for GPU computation. Instead, this paper presents several techniques that can increase accuracy during the process of self-supervised learning. The proposed techniques increase the robustness to unreliable lidar inputs. Also, they improve the depth quality for edge and sky regions based on the semantic information extracted in advance. Our experiments confirm that our model is very lightweight (2.42 ms at 1280x480) but resistant to noise and has qualities close to the latest studies.
Keywords
Depth completion; Self-supervised learning; Autonomous driving;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Ma F., Cavalheiro G. V. and Karaman S.(2019), "Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera," 2019 International Conference on Robotics and Automation(ICRA), pp.3288-3295.
2 Radosavovic I., Kosaraju R. P., Girshick R., He K. and Dollar P.(2020), "Designing Network Design Spaces," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), pp.10425-10433.
3 Scharstein D. and Szeliski R.(2002), "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," International Journal of Computer Vision, vol. 47, pp.7-42.   DOI
4 Wong A., Cicek S. and Soatto S.(2021), "Learning Topology From Synthetic Data for Unsupervised Depth Completion," In IEEE Robotics and Automation Letters, vol. 6, no. 2, pp.1495-1502.   DOI
5 Wong A., Fei X., Tsuei S. and Soatto S.(2020), "Unsupervised Depth Completion From Visual Inertial Odometry," In IEEE Robotics and Automation Letters, vol. 5, no. 2, pp.1899-1906.   DOI
6 Yang Y., Wong A. and Soatto S.(2019), "Dense Depth Posterior(DDP) From Single Image and Sparse Range," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), pp.3348-3357.
7 Godard C., Aodha O. M., Firman M. and Brostow G.(2019), "Digging Into Self-Supervised Monocular Depth Estimation," 2019 IEEE/CVF International Conference on Computer Vision(ICCV), pp.3827-3837.
8 Acuna D., Kar A. and Fidler S.(2019), "Devil Is in the Edges: Learning Semantic Boundaries From Noisy Annotations," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.11067-11075.
9 Cheng X., Wang P. and Yang R.(2018), "Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network," Proceedings of the European Conference on Computer Vision(ECCV).
10 Eigen D., Puhrsch C. and Fergus R.(2014), "Depth Map Prediction from a Single Image using a Multi-Scale Deep Network," 27th International Conference on Neural Information Processing Systems(NIPS), pp.2366-2374.
11 Yuan Y., Chen X. and Wang J.(2020), "Object-Contextual Representations for Semantic Segmentation," Proceedings of the European Conference on Computer Vision(ECCV).
12 Ma F. and Karaman S.(2018), "Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image," 2018 IEEE International Conference on Robotics and Automation(ICRA), pp.4796-4803.
13 Zhou T., Brown M., Snavely N. and Lowe D. G.(2017), "Unsupervised Learning of Depth and Ego-Motion from Video," 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp.6612-6619.
14 Guizilini V., Ambrus R., Pillai S., Raventos A. and Gaidon A.(2020), "3D Packing for Self-Supervised Monocular Depth Estimation," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), pp.2482-2491.
15 Li A., Yuan Z., Ling Y., Chi W., Zhang S. and Zhang C.(2020), "A Multi-Scale Guided Cascade Hourglass Network for Depth Completion," 2020 IEEE Winter Conference on Applications of Computer Vision(WACV), pp.32-40.