KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Depth Up-Sampling via Pixel-Classifying and Joint Bilateral Filtering

  • Ren, Yannan (School of Information Science and Engineering, Shandong University) ;
  • Liu, Ju (School of Information Science and Engineering, Shandong University) ;
  • Yuan, Hui (School of Information Science and Engineering, Shandong University) ;
  • Xiao, Yifan (School of Information Science and Engineering, Shandong University)
  • Received : 2016.11.21
  • Accepted : 2018.02.05
  • Published : 2018.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, a depth image up-sampling method is put forward by using pixel classifying and jointed bilateral filtering. By analyzing the edge maps originated from the high-resolution color image and low-resolution depth map respectively, pixels in up-sampled depth maps can be classified into four categories: edge points, edge-neighbor points, texture points and smooth points. First, joint bilateral up-sampling (JBU) method is used to generate an initial up-sampling depth image. Then, for each pixel category, different refinement methods are employed to modify the initial up-sampling depth image. Experimental results show that the proposed algorithm can reduce the blurring artifact with lower bad pixel rate (BPR).

Keywords

References

  1. C. Fehn, "Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3d-tv," in Proc. of SPIE, Stereoscopic Displays and Virtual Reality Systems XI, Vol.5291, pp.93-104, 2004.
  2. J. Kopf, M.F. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," ACM Trans. On Graphics, Vol.26 No.3, pp.1-6,2007. https://doi.org/10.1145/1276377.1276379
  3. A.K. Riemens, O.P. Gangwal, B. Barenbrug, and R.M. Berretty, "Multi-step joint bilateral depth upsampling," in Proc. of SPIE, Visual Communications and Image Processing, Vol.7257, pp.72570M, 2009.
  4. M. Liu, O. Tuzel, Y. Taguchi, "Joint geodesic upsampling of depth images," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.169-176, October, 2013.
  5. Y. Wang, A. Ortega, "A graph-based joint bilateral approach for depth enhancement," in Proc. of IEEE Conf. on Acoustic, Speech and Signal Processing, pp.885-889, July, 2014.
  6. J. Yang, X. Ye, K. Li, C. Hou, and Y. Wang, "Color-guided depth recovery from RGB-D data using an adaptive autogressive model," IEEE Trans. On Image Processing, Vol.23 No.8, pp.3443-3458, 2014. https://doi.org/10.1109/TIP.2014.2329776
  7. D. Min, J. Lu and M.N. Do, "Depth video enhancement based on weighted mode filtering," IEEE Trans. On Image Processing, Vol.21 No.3, pp.1176-1190, 2012. https://doi.org/10.1109/TIP.2011.2163164
  8. C. Jung, S. Joo and S. M. Ji, "Depth map upsampling with image decomposition," Electronics Letter, Vol.51 No.22, pp.1782-1784, 2015. https://doi.org/10.1049/el.2015.1216
  9. J. Diebel and S. Thrun, "An application of markov random fields to range sensing," Advances in Neural Information Processing Systems, Vol.18, pp.291-298, 2006.
  10. W. Hannemann, A. Linarth, B. Liu and G. Kokai, "Increasing depth lateral resolution based on sensor fusion," International Journal of Intelligent Systems Technologies and Applications, Vol.5 No.3, pp.393-401, 2008. https://doi.org/10.1504/IJISTA.2008.021302
  11. B. Huhle, S. Fleck and A. Schilling, "Integrating 3D time-of-flight camera data and high resolution images for 3DTV applications," in Proc. of 3DTV Conference, pp.1-4, May, 2007.
  12. J. Zhu, L. Wang, J. Gao and R. Yang, "Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs," IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol.32, pp.899-909, 2010. https://doi.org/10.1109/TPAMI.2009.68
  13. W. Liu, S. Jia, P. Li, X. Chen, J. Yang and Q. Wu, "An MRF-based depth upsampling: upsample the depth map with Its own property," IEEE Signal Processing Letters, Vol.22, pp.1708-1712, 2015. https://doi.org/10.1109/LSP.2015.2427376
  14. Y. Zuo, P. An, S. Zheng, Z. Zhang, "Depth upsampling method via markov random fields without edge-misaligned artifacts," in Proc. of IEEE Conference on Image Processing, pp.2324-2328, September, 2015.
  15. J. Xie, R. S. Feris and M. Sun, "Edge-guided single depth image super resolution," IEEE Trans. on Image Processing, Vol.25 No.1, pp.428-438, 2016. https://doi.org/10.1109/TIP.2015.2501749
  16. D. Ferstl, M. Ruther, H. Bischof, "Variational depth superresolution using example-based edge representations," in Proc. of IEEE Conf. on Computer Vision, pp.513-521, December, 2015.
  17. S. Kim and Y. Ho, "Fast edge-preserving depth image upsampler," IEEE Trans. on Consumer Letters, Vol.58, pp.971-976, 2012. https://doi.org/10.1109/TCE.2012.6311344
  18. S. B. Lee, S. Kwon and Y. S. Ho, "Discontinuity adaptive depth upsampling for 3D video acquisition," Electronics Letters, Vol.49, pp.612-1614, 2013. https://doi.org/10.1049/el.2013.0693
  19. Y. S. Kang, S. B. Lee and Y.S. Ho, "Depth map up-sampling using depth local features," Electronics Letters, Vol.50, pp.170-171, 2014. https://doi.org/10.1049/el.2013.3956
  20. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," International Journal of Computer Vision, Vol.47, pp.7-42, 2002. https://doi.org/10.1023/A:1014573219977