The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지
DOI QR코드

DOI QR Code

Up-Sampling Method of Depth Map Using Weighted Joint Bilateral Filter

가중치 결합 양방향 필터를 이용한 깊이 지도의 업샘플링 방법

  • Oh, Dong-ryul (Sungkyunkwan University, College of Information and Communication Engineering & Department of Human ICT Convergence) ;
  • Oh, Byung Tae (Korea Aerospace University, School of Electronics, Telecommunication and Computer Engineering) ;
  • Shin, Jitae (Sungkyunkwan University, School of Electronic and Electrical Engineering)
  • Received : 2015.03.09
  • Accepted : 2015.06.09
  • Published : 2015.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

A depth map is an image which contains 3D distance information. Generally, it is difficult to acquire a high resolution (HD), noise-removed, good quality depth map directly from the camera. Therefore, many researches have been focused on acquisition of the high resolution and the good quality depth map by up-sampling and pre/post image processing of the low resolution depth map. However, many researches are lack of effective up-sampling for the edge region which has huge impact on image perceptual-quality. In this paper, we propose an up-sampling method, based on joint bilateral filter, which improves up-sampling of the edge region and visual quality of synthetic images by adopting different weights for the edge parts that is sensitive to human perception characteristics. The proposed method has gains in terms of PSNR and subjective video quality compared to previous researches.

3D 영상의 획득을 위해 이용하는 깊이 지도는 영상의 깊이 정보를 가지고 있다. 그러나 일반적으로 고해상도이며 잡음이 제거된 고화질의 깊이 지도를 카메라로부터 직접 획득하는 것은 어렵다. 그래서 저해상도의 깊이 지도를 획득하고 업샘플링 및 전/후 영상처리를 통해 높은 품질의 고해상도 깊이 지도를 획득하는 기법들이 연구되고 있다. 하지만 기존의 연구는 영상의 질에 큰 변수로 작용하는 에지 부분의 효과적 업샘플링이 미흡하다. 그래서 본 논문은 에지 부분을 차별적으로 고려하는 인지적인 특성을 반영한 영상품질향상 연구에 초점을 맞춰 결합 양방향 필터의 가중치를 적응적으로 조절함으로써 깊이 지도와 합성 영상을 개선한 고해상도의 깊이 지도를 얻는 업샘플링 방법을 제안하였다. 제안 방식을 기존의 방식과 비교하였을 때 PSNR 측면과 주관적 품질에서 이득이 있음을 보였다.

Keywords

References

  1. W. S. Cheong, K. J. Yun, G. Bang, G. S. Lee, and N. H. Hur, "A trend of multiview 3DTV technologies and standardization," Etri. Electron. Telecommun. Trends, vol. 29, issue. 3, pp. 17- 26, Jun. 2014.
  2. Y. S. Ho, "Improve quality of the depth-map upsampling technology," Broadcasting & Technol. Mag., vol. 215, pp. 158-163, Nov. 2013.
  3. R. Cheng and K. Nahrstedt, "Empirical study of 3d video source coding for autostereoscopic displays," in Proc. ACM. Conf. Multimedia, pp. 573-576, Augsburg, Germany, Sept. 2007.
  4. C. Fehn, "Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV," in Proc. SPIE. Conf. Stereoscopic Displays and Virtual Reality Systems XI, pp. 93-104, San Jose, CA, USA, May 2004.
  5. K. Muller, P. Merkle, and T. Wiegand, "3-D video representation using depth maps," in Proc. IEEE., vol. 99, no. 4, pp. 643-656, Apr. 2011.
  6. S. B. Lee and Y. S. Ho, "Real-time eye contact system using a kinect depth camera for realistic telepresence," J. KICS., vol. 37C, no. 4, pp. 277-282, May 2012.
  7. J. H. Kim, J. W. Choi, and K. H. Son, "Depth map resolution and quality enhancement based on edge preserving interpolation," in Proc. KSBE Conf., pp. 8-10, Seoul, Korea, Nov. 2011.
  8. J. W. Choi, D. B. Min, B. S. Ham, and K. H. Sohn, "Spatial and temporal up-conversion technique for depth video," in Proc. IEEE Conf. Image Processing, pp. 3525-3528, Cairo, Egypt, Nov. 2009.
  9. X. Zhu, X. Song, and X. Chen, "Image guided depth map upsampling using anisotropic TV-L2," IEEE Signal Process. Lett., vol. 22, pp. 318-321, Mar. 2015. https://doi.org/10.1109/LSP.2014.2358883
  10. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proc. IEEE Conf. Computer Vision, pp. 839-846, Bombay, India, Jan. 1998.
  11. S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, "Bilateral filtering : Theory and applications," Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, 2008.
  12. G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, "Digital photography with flash and no-flash image pairs," ACM Trans. Graph, vol. 23, no. 3, pp. 664-672, Aug. 2004. https://doi.org/10.1145/1015706.1015777
  13. E. Eisemann and F. Durand, "Flash photography enhancement via intrinsic relighting," ACM Trans. Graph, vol. 23, no. 3, pp. 673-678, Aug. 2004. https://doi.org/10.1145/1015706.1015778
  14. J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," ACM Trans. Graph, vol. 26, no. 3, Jul. 2007.
  15. J. H. Kim, G. G. Jeon, and J. C. Jeong, "Joint-adaptive bilateral depth map upsampling," Elsevier, Signal Processing- Image Commun., vol. 29, no. 4, pp. 506-513, Apr. 2014. https://doi.org/10.1016/j.image.2014.01.011
  16. T. W. Kim, J. H. Kim, M. W. Park, and J. T. Shin, "Hybrid down-sampling method of depth map based on moving objects," J. KICS., vol. 37A, no. 11, pp. 918-926, Nov. 2012. https://doi.org/10.7840/kics.2012.37A.11.918
  17. M. S. Ko and J. S. Yoo, "Boundary noise removal and hole filling algorithm for virtual viewpoint image generation," J. KICS., vol. 37A, no. 8, pp. 679-688, Aug. 2012.
  18. T. W. Bae and Y. T. Kim, "Small target detection method using bilateral filter based on surrounding statistical feature," J. Korea Multimedia Soc., vol. 16, No. 6, pp. 756-763, Jun. 2013. https://doi.org/10.9717/kmms.2013.16.6.756