Browse > Article
http://dx.doi.org/10.5909/JBE.2020.25.6.965

Iterative Deep Convolutional Grid Warping Network for Joint Depth Upsampling  

Kim, Dongsin (Korea Aerospace University)
Yang, Yoonmo (Korea Aerospace University)
Oh, Byung Tae (Korea Aerospace University)
Publication Information
Journal of Broadcast Engineering / v.25, no.6, 2020 , pp. 965-972 More about this Journal
Abstract
Depth maps have distance information of objects. They play an important role in organizing 3D information. Color and depth images are often simultaneously obtained. However, depth images have lower resolution than color images due to limitation in hardware technology. Therefore, it is useful to upsample depth maps to have the same resolution as color images. In this paper, we propose a novel method to upsample depth map by shifting the pixel position instead of compensating pixel value. This approach moves the position of the pixel around the edge to the center of the edge, and this process is carried out in several steps to restore blurred depth map. The experimental results show that the proposed method improves both quantitative and visual quality compared to the existing methods.
Keywords
Depth map upsampling; Joint filtering; Convolutional neural network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Nasonova and A. Krylov, "Deblurred images post-processing by Poisson warping," IEEE Signal Process. Lett., vol. 22, no. 4, pp. 417-420, Apr. 2015.   DOI
2 D. Fourure, R. Emonet, E Fromont, D. Muselet, A. Tremeau, C. Wolf, "Residual conv-deconv grid network for semantic segmentation." arXiv preprint arXiv:1707.07958, 2017.
3 Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980, 2014.
4 N. Arad and C. Gotsman, "Enhancement by image-dependent warping," IEEE Trans. Image Process., vol. 8, no. 8, pp. 1063-1074, Aug. 1999.   DOI
5 A. Krylov, A. Nasonov, and Y. Pchelintsev, "Single parameter post-processing method for image deblurring," in Proc. 7th Int. Conf. Image Process. Theory, Tools Appl., pp. 1-6 Nov. 2017.
6 D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, "A naturalistic open source movie for optical flow evaluation," in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, pp. 611-625, 2012.
7 N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and support inference from RGBD images," in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, pp. 746-760, 2012.
8 H. Hirschmuller and D. Scharstein, "Evaluation of cost functions for stereo matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1-8, Jun. 2007.
9 S. Lu, X. Ren, and F. Liu, "Depth enhancement via low-rank matrix completion," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3390-3397, Jun. 2014.
10 Y. Li, J.-B. Huang, N. Ahuja, and M.-H. Yang, "Joint image filtering with deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 8, pp. 1909-1923, Aug. 2019.   DOI
11 H. Su, V. Jampani, D. Sun, O. Gallo, E. Learned-Miller, and J. Kautz, "Pixel-adaptive convolutional neural networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 11166-11175, Jun. 2019.
12 Y. Yoonmo, K Dongsin, O. Byung Tae, "Deep Convolutional Grid Warping Network for Joint Depth Map Upsampling." IEEE Access, vol. 8, pp. 147580-147590, Aug 2020   DOI
13 K. Beomjun, J. Ponce, H. Bumsub, "Deformable kernel networks for guided depth map upsampling," arXiv preprint, arXiv:1910.08373, 2019.