Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.6.906

Depth Map Completion using Nearest Neighbor Kernel  

Taehyun, Jeong (Electornics and Information Engineering)
Kutub, Uddin (Electornics and Information Engineering)
Byung Tae, Oh (Electornics and Information Engineering)
Publication Information
Journal of Broadcast Engineering / v.27, no.6, 2022 , pp. 906-913 More about this Journal
Abstract
In this paper, we propose a new deep network architecture using nearest neighbor kernel for the estimation of dense depth map from its sparse map and corresponding color information. First, we propose to decompose the depth map signal into the structure and details for easier prediction. We then propose two separate subnetworks for prediction of both structure and details using classification and regression approaches, respectively. Moreover, the nearest neighboring kernel method has been newly proposed for accurate prediction of structure signal. As a result, the proposed method showed better results than other methods quantitatively and qualitatively.
Keywords
Depth completion; depth map; kernel estimation; deep learning; deep network;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Liu, Sifei, et al. "Learning affinity via spatial propagation networks."Advances in Neural Information Processing Systems30 (2017). doi: https://doi.org/10.48550/arXiv.1710.01020     DOI
2 Ma, Fangchang, Guilherme Venturelli Cavalheiro, and Sertac Karaman. "Self-supervised sparse-to-dense: Self-supervised depth completion from lidar and monocular camera."2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019. doi: https://doi.org/10.1109/ICRA.2019.8793637     DOI
3 He, Kaiming, et al. "Deep residual learning for image recognition. " Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. doi: https://doi.org/10.1109/CVPR.2016.90     DOI
4 Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computerassisted intervention. Springer, Cham, 2015. doi: https://doi.org/10.48550/arXiv.1505.04597     DOI
5 Qiu, Jiaxiong, et al. "Deeplidar: Deep surface normal guided depth prediction for outdoor scene from sparse lidar data and single color image."Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. doi: https://doi.org/10.48550/arXiv.1812.00488     DOI
6 Eldesokey, Abdelrahman, Michael Felsberg, and Fahad Shahbaz Khan. "Confidence propagation through cnns for guided sparse depth regression."IEEE transactions on pattern analysis and machine intelligence 42.10 (2019): 2423-2436. doi: https://doi.org/10.1109/TPAMI.2019.2929170     DOI
7 Huang, Zixuan, et al. "Hms-net: Hierarchical multi-scale sparsityinvariant network for sparse depth completion."IEEE Transactions on Image Processing 29 (2019): 3429-3441. doi: https://doi.org/10.48550/arXiv.1808.08685     DOI
8 Zhang, Chongzhen, et al. "Multitask gans for semantic segmentation and depth completion with cycle consistency."IEEE Transactions on Neural Networks and Learning Systems32.12 (2021): 5404-5415. doi: https://doi.org/10.1109/TNNLS.2021.3072883     DOI
9 Nazir, Danish, et al. "SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion."arXiv preprint arXiv:2204.13635 (2022). doi: https://doi.org/10.1109/ACCESS.2022.3214316     DOI
10 Khan, Muhammad Ahmed Ullah, et al. "A Comprehensive Survey of Depth Completion Approaches." (2022). doi: https://doi.org/10.20944/preprints202205.0343.v1     DOI
11 Tang, Jie, et al. "Learning guided convolutional network for depth completion."IEEE Transactions on Image Processing 30 (2020): 1116-1129. doi: https://doi.org/10.1109/TIP.2020.3040528     DOI
12 Hu, Mu, et al. "Penet: Towards precise and efficient image guided depth completion."2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021. doi: https://doi.org/10.48550/arXiv.2103.00783     DOI
13 Yan, Zhiqiang, et al. "RigNet: Repetitive image guided network for depth completion."arXiv preprint arXiv:2107.13802(2021).  
14 Cheng, Xinjing, Peng Wang, and Ruigang Yang. "Learning depth with convolutional spatial propagation network."IEEE transactions on pattern analysis and machine intelligence 42.10 (2019): 2361-2379. doi: https://doi.org/10.48550/arXiv.1810.02695     DOI
15 Cheng, Xinjing, et al. "Cspn++: Learning context and resource aware convolutional spatial propagation networks for depth completion." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 07. 2020. doi: https://doi.org/10.48550/arXiv.1911.05377     DOI
16 Xu, Zheyuan, Hongche Yin, and Jian Yao. "Deformable spatial propagation networks for depth completion."2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020. doi: https://doi.org/10.48550/arXiv.2007.04251     DOI
17 Kim, Jiwon, Jung Kwon Lee, and Kyoung Mu Lee. "Accurate image super-resolution using very deep convolutional networks."Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. doi: https://doi.org/10.48550/arXiv.1511.04587     DOI
18 Park, Jinsun, et al. "Non-local spatial propagation network for depth completion."European Conference on Computer Vision. Springer, Cham, 2020. doi: https://doi.org/10.1007/978-3-030-58601-0_8     DOI
19 Lin, Yuankai, et al. "Dynamic spatial propagation network for depth completion."arXiv preprint arXiv:2202.09769(2022). doi: https://doi.org/10.48550/arXiv.2202.09769     DOI
20 Zhang, Kai, et al. "Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising."IEEE transactions on image processing 26.7 (2017): 3142-3155. doi: https://doi.org/10.1109/TIP.2017.2662206     DOI
21 Liu, Lina, et al. "Fcfr-net: Feature fusion based coarse-to-fine residual learning for depth completion."Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 3. 2021. doi: https://doi.org/10.1609/aaai.v35i3.16311     DOI
22 Gu, Jiaqi, et al. "Denselidar: A real-time pseudo dense depth guided depth completion network."IEEE Robotics and Automation Letters6.2 (2021): 1808-1815. doi: https://doi.org/10.1109/LRA.2021.3060396     DOI
23 Zhu, Yufan, et al. "Robust depth completion with uncertainty-driven loss functions."Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 3. 2022. doi: https://doi.org/10.1609/aaai.v36i3.20275     DOI
24 Lee, Byeong-Uk, Kyunghyun Lee, and In So Kweon. "Depth completion using plane-residual representation."Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. doi: https://doi.org/10.48550/arXiv.2104.07350     DOI
25 Chodosh, Nathaniel, Chaoyang Wang, and Simon Lucey. "Deep convolutional compressed sensing for lidar depth completion."Asian Conference on Computer Vision. Springer, Cham, 2018. doi: https://doi.org/10.1007/978-3-030-20887-5_31     DOI
26 Silberman, Nathan, et al. "Indoor segmentation and support inference from rgbd images."European conference on computer vision. Springer, Berlin, Heidelberg, 2012. doi: https://doi.org/10.1007/978-3-642-33715-4_54      DOI
27 Uhrig, Jonas, et al. "Sparsity invariant cnns." 2017 international conference on 3D Vision (3DV). IEEE, 2017. doi: https://doi.org/10.1109/3DV.2017.00012     DOI