Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Depth Map Interpolation Using High Frequency Components

고주파 성분을 이용한 깊이맵의 보간

  • Jang, Seung-Eun (Kangwon National University, IT College, Dept. of Computer & Communications) ;
  • Kim, Sung-Yeol (Gwangju Institute of Science and Technology, Dept. of Information & Communications) ;
  • Kim, Man-Bae (Kangwon National University, IT College, Dept. of Computer & Communications)
  • 장성은 (강원대학교 컴퓨터정보통신공학과) ;
  • 김성열 (광주과학기술원 정보통신공학과) ;
  • 김만배 (강원대학교 컴퓨터정보통신공학과)
  • Received : 2012.03.19
  • Accepted : 2012.05.17
  • Published : 2012.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method to upsample a low-resolution depth map to a high-resolution version. While conventional camera sensors produce high-resolution color images, the sizes of the depth maps of range/depth sensors are usually low. In this paper, we consider the utilization of high-frequency components to the conventional depth map interpolation methods such as bilinear, bicubic, and bilateral. The proposed method is composed of the three steps: high-frequency component extraction, high-frequency component application, and interpolation. Two objective evaluation measures such as sharpness degree and blur metric are used to examine the performance. Experimental results show that the proposed method significantly outperforms other conventional methods by a factor of 2 in terms of sharpness degree. As well, a blur metric is reduced by a factor of 14 %.

본 논문은 영상 보간법을 이용하여 저해상도 깊이맵을 고해상도 깊이맵으로 변환하는 방법을 제안한다. 현재의 카메라 센서는 고해상도 색상 영상을 제공하는데 반해, 깊이 측정 장치는 저해상도의 깊이맵을 주로 제공한다. 본 논문은 기존의 양선형 보간법, 고등차수 보간법, 양측 보간법을 바탕으로 깊이맵에서 추출한 고주파 성분을 적용하여 깊이맵의 선명도를 증가한다. 이를 위해, 제안 방법은 고주파 성분 추출 단계, 고주파 성분 적용 단계, 및 영상 보간 단계를 거친다. 실험에서는 다양한 깊이맵 데이터에 제안 방법을 적용하였는데, 성능검증 방법으로 선명도(sharpness degree)와 블러 메트릭 (blur metric)의 두 객관적 측정을 통해서 제안 방법이 기존 방법에 비해 선명도가 약 2배 정도 증가했음을 보여준다. 또한 블러 메트릭은 평균 14%가 감소되었다.

Keywords

References

  1. O. P. Gangwal and R. P. Benetty, "Depth map post-processing for 3D-TV," IEEE Trans. Consumer Electronics, pp. 1-2, 2009.
  2. Y. Li, L. Sun, "A novel upsampling scheme for depth map compression in 3D TV system," Picture Coding Symposium(PCS), pp. 186-189, Dec. 2010.
  3. J. Park, H. Kim, Y. Tai, M. Brown, I. Kweon, "High quality depth map upsampling for 3D-TOF cameras," IEEE International Conference on Computer Vision (ICCV), pp. 1623-6630, Nov. 2011.
  4. R. G. Keys, "Cubic Convolution Interpolation for Digital Image Processing," IEEE Trans. on Acoustics, Speech and Signal Processing, vol. ASSP-29, no. 6, pp. 1153-1160, 1981.
  5. J. Kopf, M. Cohen, D. Kischinski, and M. Uyttendaele, "Joint Bilateral Upsampling," ACM Trans. on Graphics, vol.23, no.3, article 96, pp.2:1-2:9, 2007.
  6. Q. Yang, R. Yang, J. Davis, and D. Nister, "Spatial-depth super resolution for range images," Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp.1-8, 2007.
  7. J. Diebel and S. Thrun, "An application of markov random fields to range sensing," in NIPS, pp. 291-298, 2005.
  8. E. Lee, S. Kim, Y. Jung, Y. Ho, "High-resolution depth map generation by applying stereo matching based on initial depth information," 3DTV Conference, pp.201-204, May. 2008.
  9. 박종일, "C언어로배우는실천영상처리", 성인당, 2003.
  10. H. Hou and H. Andrews, "Cubic splines for image interpolation and digital filtering,"IEEE Trans. on Acoustics, Speech and Signal Processing, vol. 26, no. 6, pp. 508-517, Dec. 1978. https://doi.org/10.1109/TASSP.1978.1163154
  11. J. Han, J. Kim, S. Cheon, J. Kim, and S. Ko, "A novel image interpolation method using the bilateral filter," IEEE Trans. Consumer Electronics, vol. 56, no. 1, pp.175-181, 2010. https://doi.org/10.1109/TCE.2010.5439142
  12. 길종인, 김만배"입체영상및CGH 화질개선을위한깊이맵처리", 한국방송공학회학술대회, pp. 27-29, Nov. 2010.
  13. M. Oskoei, H. Hu, "A survey on edge detection methods", Technical Report: CES-506, University of Essex, UK, 2010.
  14. R. Crane, "영상처리이론과실제, 홍릉과학출판사", 1999.
  15. S. Banerjee, "Low-power content-based video acquisition for super-resolution enhancement," IEEE Transactions on Multimedia, vol.11, Issue 3, pp.455-464, Apr. 2009. https://doi.org/10.1109/TMM.2009.2012925
  16. 이아영, 김희창, 정제창, "이웃픽셀값을고려한적응적3차보간법", 방송공학회논문지, 제15권, 제3호, pp. 362-367, 2010.
  17. http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/
  18. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," International Journal of Computer Vision, vol. 47, no. 1-3, pp. 7-42, 2002. https://doi.org/10.1023/A:1014573219977
  19. Q. Thu and M. Ghanbari, "Scope of validity of PSNR in image/video quality assessment," Electronics Letters, Vol 44, Issue 13, pp.800-801, Jun. 2008. https://doi.org/10.1049/el:20080522
  20. C. Tsai, H. Liu, M. Tasi, "Design of a scan converter using the cubic convolution interpolation with canny edge detection," 2011 International Conference on Electric Information and Control Engineering (ICEICE), pp. 5813-5816, Apr. 2011.
  21. P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, "Perceptual blur and ringing metrics: application to JPEG2000," Int. Workshop Multimedia Signal Processing, pp.403-408, Oct. 2008.